A robot controller developed by Peking University and the Beijing Academy of Artificial Intelligence separates planning from hand movements: one model looks ahead while another updates the robot's actions using fresh camera views. Called LiMA, it matched or exceeded four competing systems on task completion in six laboratory tasks, according to the team's September 23 preprint.
A robot making a sandwich must keep track of the whole job while adjusting its grip as objects move. Generating a detailed prediction of the next scene at every movement update takes computing time. LiMA keeps the longer-range plan available between planning updates, allowing a smaller model to concentrate on the robot's next movements.
A shared plan guides more frequent movement updates
The planning model, named Future Dreamer, predicts how the scene and the robot's actions should develop. The smaller Motion Refiner uses that plan alongside current camera images and joint positions to generate a short sequence of arm-and-finger commands. In the tested schedule, the Refiner updates four times for every new Dreamer plan.
A method called Latent Schrödinger Bridge Coupling uses the Dreamer's plan as the starting point for generating movements. The Refiner learns to turn that rough guidance into more precise actions, with fresh observations helping it adjust to the current scene.
Higher completion rates did not make LiMA the fastest controller
The researchers trained the system with 100 human-operated demonstrations per task, then tested it on two robot arms with five-fingered hands and three cameras. Each system had 20 trials per task. LiMA completed 16 rice-cooking trials and 14 sandwich-making trials, compared with 14 and 13 for NVIDIA's GR00T N1.6.
Across all six tasks, LiMA's completion rates ranged from 50% to 90%, tying or beating the four comparison systems in each task. Those small laboratory trial sets do not establish reliability in routine use; LiMA completed just 10 of 20 package-assembly trials.
Generating commands was faster than with some competitors, but slower than with others. On the same NVIDIA H100 processor, LiMA averaged 325 milliseconds to produce a sequence of 32 actions, compared with 600 milliseconds for Cosmos-Policy. VPP and GR00T N1.6 were faster than LiMA. The researchers held batch size and numerical precision constant and did not apply model-specific speed optimizations.
The 325-millisecond average spreads the cost of the less frequent planning updates across the movement updates. It measures the time to generate a whole action sequence, not the delay for a single movement or the time to finish a task.
LiMA still depends on what its cameras can see. The authors report poorer performance when objects are heavily obscured or scenes offer little visual variation. They propose adding touch sensing to provide information that cameras miss.
Sources
- LiMA preprint, arXiv:2609.28431 (posted September 23, 2026; CC BY 4.0)
- LiMA project page
Lead image: Figure 4 from the LiMA preprint by Ning Chen et al., licensed CC BY 4.0.