Seven AI models passed only six of 168 attempts at engineering tasks that required work to move between software tools, according to EngiWorld, a new benchmark paper.
EngiWorld defines 1,301 expert-curated tasks across six engineering fields and 26 software platforms. The researchers tested seven models on a 300-task subset. Within that sample, 24 tasks required multiple programs; each model attempted the same 24, producing 168 model-task runs. The six passes refer to those runs, not 168 different workflows.
The benchmark checks saved files rather than relying on an agent's claim that it is done or on a plausible-looking screen. Its verifiers reopen outputs to inspect properties such as geometry, physical feasibility, circuit connections, simulation results and consistency across stages. A pass means a saved output met the paper's checks.
A recent CADWorld study tested 200 workflows in FreeCAD, a computer-aided design application. EngiWorld extends evaluation across more engineering fields and software platforms, but the studies use different tasks and their scores are not comparable. EngiWorld is an author-run preprint benchmark, not a measure of deployed engineering teams.
Sources
- EngiWorld paper, September 29, 2026.
- CADWorld paper, cited for scope context only.
- Illustrative image: “FreeCAD 1.0 Light Assembly Example,” by Maxwxyz, CC BY 4.0 via Wikimedia Commons; not from EngiWorld.
