Stanford University researchers have built a memory system that lets a mobile robot reuse information gathered during earlier jobs instead of inspecting the same space again. In a continuous simulation of 25 household tasks lasting more than three hours, MessyMem recorded 80.0% task progress, compared with 65.2% for its strongest stripped-down version and 51.1% for the strongest external baseline. The paper was submitted on September 14 and accepted at the 2026 Conference on Robot Learning. It targets a practical limitation in current robot planners: experience often disappears when one task ends.
A mobile manipulator combines a wheeled base, an arm and cameras so it can move through rooms and handle objects. Existing systems commonly keep a map, a short history or a video archive. A map may store the location of a drawer while missing the fact that the drawer is locked. Video preserves details, but hours of footage are difficult to search. The result is repeated work, such as opening cabinets that the robot has already checked.
MessyMem stores three kinds of evidence in one searchable record. It builds a 3D scene graph, a structured list of objects and their locations, then adds properties discovered through action, such as whether a cabinet opened or a grasp failed. It also saves selected camera frames that preserve details such as labels and patterns. When a new instruction arrives, a retriever finds relevant objects and images, a vision-language model chooses the next action, and the outcome updates the same memory before planning continues.
The researchers isolated the memory system from the rest of the robot stack. In two short RoboCasa365 simulation tests, every method used the same planner, perception system and action commands, and each method ran 50 trials. A longer kitchen test presented 25 consecutive finding, matching, manipulation and recall tasks across 10 cabinets and more than 60 objects without resetting memory. The team then used a TidyBot++ robot for office-drawer searches and sock matching. A human operator approved each physical action for safety and confirmed the final result.
The largest gap appeared in the long simulation. MessyMem's 80.0% task progress was 14.8 percentage points above a version without linked camera keyframes and 28.9 points above RoboEXP, an external scene-graph baseline. In the cluttered-pick test, the full system reached 84%, while the version without keyframes reached 58%. In six real-world sock-pairing trials, MessyMem scored 1.00 task progress; the two versions missing one memory component each scored 0.67.
The tests show why long-term robot memory needs both compact structure and recoverable visual detail. A location map tells the planner where to go, while stored interaction results can prevent a repeated failed action. The evidence remains early. Object detection used a predefined category list, simulated navigation relied on scripted helpers, and the small real-world trials were operator-gated. Longer deployments must test changing rooms, human actions and stale memories, including whether an incorrect memory causes a robot to skip a check that safety requires.
Related coverage
Sources
- Anuva Banwasi and colleagues, “MessyMem: Learning-from-Doing Memory for Mobile Manipulation,” arXiv, September 14, 2026. Abstract, authorship, submission date, acceptance note and principal results.
- Full paper. Memory architecture, experimental controls, task definitions, results and limitations.
- MessyMem project page. Videos and supporting material supplied by the research team.
- Lead image: Jose Luis Rivero via Wikimedia Commons, Apache License 2.0. The image is illustrative and does not show the reported experiment.
