Researchers report that a robot agent’s manipulation success rose from 92.6% to 96.5% when it could reopen short video clips from a task demonstration during simulated runs. The method, Recursive Video In-Context Learning (RV-ICL), leaves the pretrained vision-language-action policy — the model that turns camera views and commands into robot actions — unchanged and adds video retrieval for the agent to use while it acts.
A full video can crowd an agent’s context when it is reconsidered at every step, while a few still frames can miss a grasp or release. RV-ICL indexes one demonstration per task at four levels: task-wide keyframes, phases, moments around events and clips of up to 20 frames. The agent reads the overview before acting, then requests a clip for its current subgoal or after a step fails. Since the demonstration was recorded in another layout, it uses its own camera views to find objects rather than copying their recorded positions.
On LIBERO-PRO, eight benchmark cells with 50 simulator episodes each, success rose from 92.6% with HarnessVLA alone to 96.5% when RV-ICL was added on top of HarnessVLA. In a separate LIBERO-Plus test of 120 task variants across seven kinds of perturbation, success rose from 86.7% to 95.8%. The authors report simulator goal completion; these scores are not physical-robot results.
The added video access took time and context. Across the 120 LIBERO-Plus variants, median episode time rose from 8.3 to 14.6 minutes and median input tokens per episode from 1.19 million to 1.71 million. The tests used one robot and one VLA policy in LIBERO; the authors say tasks such as pushing, wiping or pouring need a different way to identify subtask boundaries, and other robot designs remain untested.
Sources
- arXiv paper: Recursive Video In-Context Learning for Agentic Robot
- Full paper text
- Authors’ project page and demonstration
- Lead image: Figure 1 by the authors, reproduced under CC BY 4.0.