A preprint posted on arXiv on Oct. 5 tested whether a robot would switch to another object when researchers changed only its instruction. Across the tested settings, policies usually moved toward the object named in the original command, even when the new target was visible and reachable.
The study compared two vision-language-action (VLA) policies, which turn camera and text input into robot actions, with two world-action models, which learn how scenes change as robots act. For the main target test, researchers held the scene and robot state fixed and changed only the object named in the instruction, choosing another visible, reachable object.
In LIBERO-Object simulation, the four policies committed to approaching the original target in 86.5% to 98.3% of trials; approaches toward the newly requested target ranged from 0.5% to 13%. In six physical scenes, the new target was picked up in 5.8% of trials with π0.5 and 3.3% with NVIDIA's GR00T N1.7. Pickup of the original target succeeded in 74.2% and 90.8%, respectively.
Changing the wording affected task success and the actions the policies predicted before moving, so they did not simply ignore language. A simple classifier could identify the newly requested target from internal signals in both VLA policies, even when their actions favored the original object; the researchers' analysis points to possible reasons but does not prove what caused the mismatch.
The physical tests cover six scenes and two policies, and the paper is a new preprint. These results show a failure mode in those settings, not how often deployed robots would mishandle changed commands. Ordinary task-completion scores alone can miss whether the robot followed the latest request or repeated behavior associated with a familiar scene.
Sources
The study (arXiv preprint) · Physical Intelligence's OpenPI repository · Image: Nenad Stojković via Wikimedia Commons (CC BY 2.0; used unchanged as an illustration, not an image of the tested systems).