An extra step that rewrites a command before a robot acts raised a frozen robot-control model’s task-success rate by 16% to 27% relative in simulation, according to a paper posted Oct. 7. The gain was measured on 12 held-out tabletop tasks and varied with both the wording and the rewrite model.
A vision-language-action (VLA) model turns a camera view and a text instruction into robot actions. But a controller trained around particular words can respond differently when a person uses a synonym or changes a phrase, even when the requested task stays the same.
To address that gap, the researchers scored 5,453 instruction-and-task pairs across 228 Bridge tasks. They used simulator success where rollouts were available; for recorded demonstrations that could not be simulated, they measured how closely the controller’s predicted gripper movements matched expert actions, using that error as a proxy for success. A language model distilled the scores into 10 to 20 rules, such as keeping “plate” instead of substituting the near-synonym “dish.” Before a new episode, another model applies the rules once to rewrite the instruction; the robot controller’s weights stay unchanged.
On 12 sealed SIMPLER tabletop tasks across 24 fixed scene layouts, the rewrite improved success by 16.1% to 26.5% relative to passing the original wording directly to the controller. The comparison covered adversarial, human-written and model-generated instructions, and the improvement appeared across three rewrite models: Claude, Gemini and Qwen. The gains were strongest on tasks outside the controller’s fine-tuning set.
A second test paired a different controller, π0.5, with the LIBERO benchmark. Success on 10 tasks included in fine-tuning rose from 93.6% to 97.8%; on 10 tasks outside fine-tuning, the change was not measurable (p>0.4). The authors tested two VLA model families, only in simulation. Rewriting also adds two model calls before execution: the paper reports about 2.5 seconds to prepare a reasoning trace and another 1 to 17 seconds for the rewrite, depending on the model.
Photo credit: Nenad Stojković via Wikimedia Commons, CC BY 2.0. The photograph is illustrative and is not from the study.
