Researchers tested whether coding agents can turn simulator experiments into robot-control programs that solve previously unseen starting arrangements of the same task. In a Sept. 24 preprint, they report that the resulting programs outperformed hand-engineered planners on average in the simulated settings where both could be tested.
Task-and-motion planning, or TAMP, combines two linked decisions: what an agent should do, and whether a robot can carry out the required motion without violating geometric or physical constraints. A planner may need to choose an object, find a collision-free path and adapt when moving one object changes what is reachable next. The researchers asked whether a general coding agent could discover reusable strategies without being given the usual hand-built planning components.
The team tested Codex with GPT-5.6 Sol, Claude Code with Opus 5 and Codex with GPT-6 Astra in 28 simulated environments from the KinDER and PDDLStream benchmarks. In the main setup, agents received a task description and access to a simulator, but not the environment’s source code or a custom TAMP abstraction. They could choose experiments, write Python tests, inspect outcomes and revise a program.
Each finished program was frozen before evaluation and tested on 100 previously unseen initial configurations; the language models were not called during those tests. Across seven synthesis methods, the researchers evaluated 980 programs in 98,000 episodes.
On the 16 environments with a hand-engineered planner baseline, GPT-5.6 Sol averaged 56% success, Opus 5 averaged 82% and GPT-6 Astra averaged 95%; the planners averaged 47%. These are mean benchmark success rates, not a claim that every program solved every task.
The environments were simplified simulations that exposed each object’s position and geometry directly, so the study did not test visual perception or control on physical robots. Some dynamic three-dimensional tasks, including sweeping or pouring many small objects, remained difficult. The authors note that the models’ training data are undisclosed, so prior exposure to benchmark code cannot be ruled out.
Sources
Image: Lexington Medical, Inc., via Wikimedia Commons (CC0). Illustrative manufacturing photograph; the study used simulated robots.