A language-model planner used fewer simulator measurements than a free-form baseline in tests that asked AI systems to infer equations from physics simulations, a new arXiv preprint reports. The simulations were noiseless; the study did not test measurements from real physical systems.
The workflow asks a language model to propose candidate equations, then a planner chooses inputs for a hidden simulator. After the measurements, a fresh model prompt writes the final equation. In the interpreter-enabled comparisons, both approaches could also use Python for analysis.
Across 576 trials covering 12 physics-style tasks and eight configurations, the planner used an average of 8.6 measurements per medium-difficulty run with GPT-4.1-mini, compared with 22.5 for the baseline. A magnitude-based root mean squared log error (RMSLE) averaged 0.202 with the planner and 2.514 with the baseline; lower scores mean closer output magnitudes on the test grid. In a targeted GPT-4.1 comparison on medium tasks, the planner used 8.9 measurements versus 43.0, with RMSLE scores of 0.149 versus 0.626.
These were comparisons between complete protocols. Both approaches could use Python for analysis. The free-form baseline had 10 turns and could request variable-size batches; its suggested cap of 20 measurements was not enforced. The planner explored for eight turns, then capped later requests at four measurements before a separate model prompt wrote the final equation. The results do not isolate query selection as the cause of the difference, and fewer simulator calls do not establish lower compute or API costs.
The RMSLE score compares output magnitudes and does not establish that the system recovered the correct equation; exact symbolic recovery varied across tasks. Randomly choosing input points was competitive with choosing points where candidate predictions disagreed. Each model, module and configuration cell had only two runs; the authors call for more trials and comparisons with matched measurement budgets.
Sources
Image credit: Preply.com Images via Wikimedia Commons, CC BY 2.0; unmodified. The photo is illustrative and is not from the study.
