Researchers built RAPID, a coding-agent system that turns one visual demonstration and a task description into robot routines for new scenes. They tested its programs across eight simulation tasks and then on a physical robot arm.
For the eight main tasks, the team did not provide ready-made actions such as pushing or flipping. Objects could not be picked up directly, so the robot had to move them into graspable positions. RAPID reconstructs each scene in a physics simulator, infers the task's goal and creates motion routines. The coding agent tests these routines across simulated scene changes and revises them when they fail.
Rather than replaying the demonstrator's exact path, each routine describes how objects should relate to one another—for example, where a box must sit relative to a corridor. RAPID then calculates a path for the geometry of each new scene. The strategy stays fixed across the test scenes, while the system maps its object roles to each scene in a few seconds.
In simulation, the researchers tested eight tasks across 50 new scenes per task, varying object pose, appearance, size, shape and material. RAPID averaged 75.9% success, with a 2.4-point standard deviation across three trials. Without generating scene variations for verification, its average fell to 53.2% (7.1-point standard deviation). In a separate test on a Franka Research 3 arm, RAPID succeeded in four to nine of 10 scenes per task, compared with zero to three for another robot-programming system, CaP-Agent0.
The results show that a program built from one example can handle some changes in objects and layout, in simulation and in these physical tests. The evidence is limited to eight tabletop tasks on one robot arm; the paper is a preprint, and the project page lists code as coming soon.