Researchers report that an AI-built robot system carried eight of the 10 object-handling tasks it completed in simulation onto real robot arms. Across all 18 tasks in the study, it succeeded on eight in hardware and 10 in randomized simulation, according to an Oct. 7 preprint from teams at Sun Yat-sen University and the Hong Kong University of Science and Technology (Guangzhou).
The system, called Agentic Real-to-Sim-to-Real (Agentic RSR), starts with a video of a robot workspace, a task description and a model of the robot. It uses the robot’s known geometry to estimate the scene’s scale, then an AI agent revises a 3D reconstruction against reference images. A separate agent checks whether simulated objects and surfaces can support the requested interactions in MuJoCo, a physics simulator.
Agentic RSR then writes a robot-control program and tests it in stages. It first uses object positions that would not be available during deployment; later tests remove those coordinates and rely on camera observations, then vary object positions, camera setup and action execution. The policy and a record of useful and failed behaviors move to the real robot, where new observations and safety checks guide the next actions. The paper says the experiments used Codex agents powered by GPT-6 Astra with high reasoning effort.
Those results mean the system reproduced 80% of its simulation successes and succeeded on 44.4% of all 18 tasks in hardware.
The study is small, and its comparison with other robot systems is not a matched hardware test: the authors say some baseline scenarios did not align with the same indexed tasks. They also report that the closed GPT-6 Astra service behaved differently across devices and invocation setups, but did not measure that variation under a controlled protocol. The authors say they plan to release the code and reconstructed scenes publicly; neither is supplied with this preprint.