Researchers at Shanghai AI Laboratory report a 100-hour dataset for robot learning that aligns first-person video with reconstructed body motion, finger movement and touch. The first downloadable release is much smaller: 18 recordings totaling 66 minutes, behind an access request, according to the dataset page.
The collection is meant to show how people move while handling objects: walking to a work surface, changing stance, reaching with both hands and feeling contact. A camera alone records what a person sees but can miss the body and hand motion behind that view. EgoExplorer synchronizes these signals so robot-learning researchers can study them together.
A wearable rig estimates the unseen parts
The paper describes a head-mounted camera and five trackers placed at the head, hands and ankles. Gloves record finger articulation; one glove setup also measures touch. The team aligns the streams on a shared timeline and annotates the steps in each task. Its current 18-episode sample has all four streams in every recording. Across the paper's larger set of regular recordings, however, measured touch appears in 41%, so the modalities are not present uniformly throughout the full collection.
Five trackers do not directly measure every body joint. The team first calibrates each participant's body proportions and how the trackers sit on the body, separating fixed mounting offsets from actual joint movement. A temporal model then estimates the untracked joints, with further constraints for skeletal motion, self-intersection and contact with the floor.
On the paper's evaluation subset, the team reports 18.0 millimeters of body-point error, compared with 64.1 millimeters for a fine-tuned AvatarPoser baseline using five tracker anchors. The references were fitted to tracker measurements, making this a test of human-motion reconstruction, not a robot's task success.
The preview is not a robot trial
The authors' preliminary learning experiments combine video, motion and touch, and retarget two human recordings to a Unitree G1 controller in simulation. They do not report a physical humanoid trial. The paper says real-world learning benefits remain unverified, and notes that some joints must be inferred and tactile sensing can be unreliable.
The dataset page lists the release under a noncommercial license and requires an access request before files can be downloaded. It says the full collection is being opened in stages; researchers can join a waitlist for that release.
Sources
- EgoExplorer preprint, version 1 (October 13, 2026)
- Official dataset page, release status and access terms
- Illustrative image: Mbrickn, “2010 Motion Capture Suit,” Wikimedia Commons, dedicated to the public domain under CC0 1.0. The pictured suit is not the EgoExplorer rig.
