Researchers at the University of Illinois Urbana-Champaign, UC Berkeley and Northwestern University have taught a robot hand to predict how contact will change during an action. Their DexTacWAM system achieved a mean task score of 70.6 out of 100 across six two-handed manipulation tasks, compared with 38.0 for the strongest baseline. Some tasks award partial credit for completed steps.
Cameras can show where a robot hand and an object are, but they often miss the pressure, slip and grip stability that decide whether an action succeeds. Existing tactile policies can react to the force measured at the current moment. DexTacWAM instead treats touch as part of the future state that a world model must predict alongside video.
Ten fingertips become two compact touch views
Each fingertip sensor produces a small grayscale image of a deformable marker grid. Contact bends that grid, turning pressure and shear into a spatial pattern. DexTacWAM sends each pattern through a frozen visual encoder, then uses finger identity, hand pose and recent time steps to compress five fingertip streams into one representation for each hand.
The two hand representations join a head-camera view inside a pretrained video diffusion model. That model predicts how the visible scene and the distributed contact patterns will evolve. An action module reads those predictive features and outputs the next robot motion. It is also trained to predict contact force and the robot's next state. The force target supplies an extra learning signal; the controller does not take force readings as an input at runtime. Compressing the fingertips reduces the model from 11 separate views to three while preserving which finger made contact.
The researchers adapted the tactile encoder on 488 bimanual episodes, about four hours of interaction, while keeping the visual encoder frozen. They then fine-tuned the world model and trained the action module with roughly 100 demonstrations for each task. This reused a visual representation for tactile images instead of building a touch model from scratch.
Prediction beats current-touch conditioning in the tested tasks
The evaluation covered cube placement under occlusion, handover, wiping, tong use, bowl separation and bottle-cap unscrewing on one 22-degree-of-freedom robot platform. Each method ran 20 real-world trials per task. Handover earned credit separately for picking up, transferring and placing the cube; wiping earned credit for five stages. The other four tasks required full completion to score. DexTacWAM led on all six, including a 60 score for tong use against 10 for the strongest baseline, where force transmitted through the tool was difficult to infer from cameras.
A controlled ablation kept the same tactile features, observations and action module but removed tactile prediction from the world model. Across four tasks, the average fell from 74.7 to 26.6. That gap is stronger evidence for the paper's central claim than the headline comparison alone: the benefit came from modeling how contact evolves, not simply from giving the policy a touch sensor.
On one RTX 4090, compressing the touch inputs cut training iteration time from 3.87 to 1.71 seconds and inference time per action chunk from 363.0 to 281.6 milliseconds. Those efficiency numbers describe this implementation and hardware, so they should not be read as a general speedup for every tactile controller.
The study remains narrow in two important ways. It uses one robotic hand design and one vision-based tactile-sensor design, leaving transfer to capacitive, magnetic or other sensors untested. It also trains only on successful demonstrations, giving the model little evidence for recovering after a slip or failed grasp. DexTacWAM shows that predicted contact can improve contact-rich control in this setup; it does not yet establish a general touch model for unfamiliar robots or failures.
Sources
- Yuan et al., “DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation,” arXiv:2609.24976 (submitted September 21, 2026)
- DexTacWAM project page, task videos, method details and evaluation tables
- Lead image: “Human-robot” by Mohamed Hassan, a generic illustration rather than the research platform, dedicated to the public domain under CC0 1.0 via Wikimedia Commons.