A research model that predicts what a robot will see and feel learned better contact forecasts after its developers added human glove recordings while keeping real-robot training fixed at five hours. In the new DexTouch-WM preprint, Contact-F1, a score that penalizes both missed contacts and falsely predicted ones, rose from 0.55 to 0.71 at the largest human-data setting. The result suggests that human touch can supplement scarce robot interaction when both domains use the same sensing layout.
Robot touch data are expensive because each trajectory occupies a physical machine and is tied to its sensors. The researchers instead collected about 100 hours of bimanual human interaction across 50 everyday tasks, then combined it with five hours from six robot pretraining tasks. Each human glove and robot hand presented the model with the same 320 pressure-sensing points per hand. Tracked human wrist and finger motions were also retargeted into the robot's action coordinates. The held-out robot tasks did not appear in the human pretraining set, so the scaling test measured transfer across tasks rather than rehearsal of the same actions.
DexTouch-WM predicts what the camera and both tactile maps will show after a proposed action sequence. A pretrained video generator handles images while a smaller tactile branch encodes fingertips and palms as separate anatomical regions. The two branches exchange information through joint attention. The tactile branch predicts changes from the first pressure reading instead of rebuilding the static background, and action signals reach video and touch at their different native rates. This sequence lets the model connect hand motion, object motion and changing pressure without treating disconnected pads as neighboring image pixels.
The scaling curve matters as much as the endpoint. With robot data held constant, adding 10 hours of human interaction barely changed Contact-F1, from 0.551 to 0.554. It reached 0.659 with 50 hours and 0.706 with 100 hours. These are internal comparisons on the paper's held-out robot episodes, not a benchmark against other tactile world models, and the sharpest contact gains appeared only after substantially more human data.
The authors then tested whether the model could stand in for the robot when comparing policies. Across three policies and four tasks, a task-adapted model trained with 100 robot and 100 human demonstrations achieved a mean correlation of 0.844 with real-robot policy scores, where a value nearer one means the predicted and real scores tracked one another more closely. The version adapted with 200 robot demonstrations reached 0.646. The mixed model also reduced the mean maximum rank violation, a penalty for reversing policy order, from 0.025 to 0.017. Both versions still misordered two policies on the upright-bottle task, so the average improvement did not remove task-specific failures.
Generating training data was harder than evaluating policies. The team replaced half of each policy's 200 real training trajectories with 100 world-model rollouts, then scored the trained policies on the physical robot. Averaged across four tasks, five of the six policy-and-generator combinations performed worse than their all-real baseline. One robot-adapted generator produced a small average gain for X-VLA, but the human-mixed generator drove that policy's bottle score to zero. Visually plausible forecasts were therefore useful for ranking policies more often than they were reliable substitutes for real training experience.
The present result is a data-transfer proof on one robot hand, one shared pressure-array design and a small set of policies and tasks. It does not show that arbitrary human touch recordings transfer across different sensor geometries, and the arXiv entry does not provide public code or the collected data.
