Robot policies trained on demonstrations can take timing cues from people performing the same task, instead of using one preset speed multiplier. In a new arXiv preprint, researchers at Japan’s National Institute of Advanced Industrial Science and Technology (AIST) report that their EgoSpeedUp method succeeded in 33 of 40 trials across two tabletop tasks, compared with 23 of 40 for a nominal behavior-cloning policy trained on the original robot actions. The tests covered rope straightening and moving a stack of blocks.
During training, EgoSpeedUp lines up the same ordered stages—approach, grasp, manipulation and release—in human videos and robot demonstrations of a task. For each robot phase, it compares the phase length with matching human examples and averages the ratios across those videos. When people perform a phase faster, the ratio leads the training process to select robot action targets from later points in that phase of the demonstration, teaching a faster target tempo. Standard behavior cloning then trains on these adjusted targets paired with the original robot observations; human video and a tempo lookup are not needed at run time.
The results differed by task. In rope straightening, EgoSpeedUp succeeded in 16 of 20 trials, up from 12 of 20 for the nominal policy. Among successful trials, mean completion time fell from 14.76 to 5.60 seconds. When moving a tower of blocks, it succeeded in 17 of 20 trials versus 11 of 20, while mean time among successful trials fell from 14.48 to 12.89 seconds.
The paper reports 20 evaluation trials per method and task: two policy seeds, each tested from 10 initial poses. The reported time averages include successful episodes only, so they exclude attempts that failed. Training used 80 teleoperated robot demonstrations and 40 egocentric human videos for each task; all comparison methods used the same robot demonstrations.
The approach assumes that human and robot demonstrations can be matched to the same ordered sequence of phases and assigns each robot training episode a deterministic target tempo for each phase. The authors say that repeated or branching tasks, and speed changes within a phase, remain open problems. The results come from two tabletop tasks; transfer to less structured tasks and more varied scenes has not been tested.
Sources
Original paper: EgoSpeedUp (HTML) · arXiv record · Photo by Shixart1985 (credit: Nenad Stojković), via Wikimedia Commons, CC BY 2.0.
