An Oct. 2 arXiv paper reports that a two-armed robot performed precision tasks using a fixed stereo camera and software-adjusted views instead of wrist-camera inputs. EyeRobot 2.0 shifts image detail toward the object the robot needs to handle as the task changes.
Wrist cameras give a gripper close-up views of its work, but a held object can block the lens. EyeRobot keeps its stereo camera fixed between the arms. Software remaps the two stereo views around a selected 3D point, giving the controller more image detail there without moving the physical camera.
EyeRobot separates choosing what to see from moving the arms. A target selector picks the object that matters at each step; a gaze policy centers the stereo views on it. The gripper controller receives image crops with extra detail around that point, alongside the robot's joint and pose data. Trained on teleoperated demonstrations, it predicts the next arm movements. Reinforcement learning rewards the target selector for views that help the controller make those predictions; the system needs no human gaze labels.
Across seven physical tasks, the authors' project page reports a mean full-task success rate of 67%, compared with 52% for stereo plus wrist cameras and 27% for passive stereo alone. The mean weights the seven tasks equally; it is not a pooled count of every robot attempt.
The comparison changed with wrist-camera visibility. When those cameras had clear views, EyeRobot averaged 69% versus 64% for the stereo-plus-wrist-camera policy. When grasped objects blocked the cameras, the rates were 48% versus 22%.
The researchers collected teleoperated demonstrations with both stereo and wrist cameras, and trained the comparison policies on those same demonstrations and test positions. EyeRobot needed only stereo-camera inputs while performing the tasks. The evaluation used one two-arm YAM robot and task-specific policies; it does not show one model generalizing across tasks or robots, or controlling head and neck movement.
Sources