A new locomotion system lets a humanoid robot use two depth cameras to spot stairs ahead, slow down for the climb and return to its commanded pace afterward. Researchers at Caltech demonstrated the system on a Unitree G1, which climbed and descended real staircases and also walked, ran and jumped onto and off boxes.
The study, posted as an arXiv preprint on Sept. 25, divides the job between two learned controllers. A motion generator uses a forward-facing depth camera to look ahead and a downward-facing camera to read the ground near the feet. It proposes a path for the robot’s whole body; a faster tracking controller turns that plan into joint movements.
That look-ahead lets the robot adapt even when its joystick keeps sending the same speed request: it slows for terrain it cannot cross safely at that pace, then resumes the requested speed afterward. In hardware tests, the same system traversed six differently shaped staircases and climbed one flight of 15 consecutive steps, alongside indoor and outdoor walking, running and box-jump demonstrations.
The team fine-tuned the motion generator in simulation using reinforcement learning, while keeping the tracking controller fixed. Because each planned motion contains thousands of values, adding random changes to every value would make useful exploration difficult. The researchers instead varied the generator’s inputs, used a learned critic to score simulated paths, then trained the generator to favor higher-scoring ones.
Across the authors’ simulated terrain and task tests, fine-tuning raised successful traversals by as much as 25 percentage points over the original generator. In one stairs setup, the percentage of times the robot chose the movement people had labeled correct rose from 0% to about 80%; researchers assessed 50 simulated rollouts per policy and terrain. These are simulation results, separate from the real-robot demonstrations.
The controller handles movement, not full navigation. A Bluetooth joystick still supplies the robot’s speed command, and depth images show shape rather than meaning: the system cannot tell whether a box-shaped object is safe to jump onto. The authors say navigation and safety or obstacle-avoidance systems still need to be integrated. The paper is an arXiv preprint, and its results are reported by the authors.
Sources
- Caltech researchers’ paper, “Generate, Track, Improve” (arXiv:2609.31577) · full text
- Image: Sayanesy, “Unitree G1 at Uav expo 2024,” Wikimedia Commons, CC0 1.0. The photo shows the robot model, not the study demonstration.
