A UC Berkeley team trained a diffusion model to generate neural-network controllers for a Unitree G1 humanoid, then ran sampled networks on the physical robot to walk, dance and spin-kick without fine-tuning. Instead of generating a sequence of movements, the model generated the controller that maps robot observations to joint commands. The reported success rates, however, come from simulation; the authors give no real-world success-rate estimate.
Robot-learning systems commonly generate actions or short action sequences for a robot to follow. The Berkeley group instead models the learned weights and biases that define a controller. Once sampled and decoded, each network runs as a standard policy on the robot; the diffusion model is no longer in the control loop.
The model learns a distribution of existing controllers
The researchers trained separate models for walking, dancing and spin-kick motions. For each motion, they ran 750 reinforcement-learning jobs with different random seeds and kept eight converged networks from each run, producing 6,000 controller examples. Each was a three-layer neural network with roughly 30,000 to 40,000 parameters. A linear autoencoder compressed the weights into a 2,048-number representation, and a diffusion transformer learned to generate samples in that smaller space before they were decoded back into controllers.
In simulation, the team tested 100 generated networks per motion across 1,024 parallel environments. Mean success rates were 99.3% for walking, 94.1% for dancing and 94.3% for spin kick. Networks from the reinforcement-learning dataset averaged 99.9%, 99.8% and 99.7%, respectively. The lower generated averages and wider spread for dancing and spin kick show that performance varied across samples. The paper also reports results for the best 90% of generated networks; that selected subset is not the same as the model’s average reliability.
One robot shows three motions, but reliability and transfer remain unproven
The paper and project page show generated controllers running on a physical Unitree G1 without additional fine-tuning. They demonstrate walking, dancing and a spin kick, but do not report an aggregate count or success rate for real-robot trials. The simulation percentages therefore should not be read as physical-robot success rates.
The authors also found that the first layer of the controllers gave relatively large weights to inputs associated with the waist and hip joints. They suggest this pattern may reflect a “kinematic prior,” or structure related to the robot’s body, because a similar pattern appeared in networks trained through both reinforcement learning and imitation learning. They describe the interpretation as preliminary; it has not yet improved policy generation or control performance.
The study covers one humanoid platform and three motions, with a separate generator for each. The work is a preprint under review, and the project page says code is coming soon. It shows that diffusion can produce working controller weights in this setting, but not yet a general system that can create new behaviors on demand or transfer them to other robots.
Sources
- Full paper (arXiv:2610.20606v1)
- Project page and robot demonstrations
- Illustrative photo and license information
Photo credit: RuinDig/Yuki Uchida, CC BY 4.0, via Wikimedia Commons. This event photo is illustrative and is not from the study.