A navigation policy can find its way near a requested object and still fail the task by walking past it. In an arXiv preprint submitted Oct. 16, researchers trace that gap to a specific weakness: the model often cannot tell that the goal is visible or estimate how far away it is.
The work, called StopWatch, tests object-goal navigation: an agent must find an object category, such as a vase, in an unfamiliar indoor scene and stop near an instance. The authors report that an end-to-end vision-language policy came within 25 centimeters of the goal in 76.3% of 3,000 HM3D-OVON episodes but succeeded in only 52.3%. When given ground-truth information about which objects were visible and the distance to the goal, the same policy’s success rose to 89.3%. The deployed system instead must predict that information from camera images.
Two models separate seeing from acting
StopWatch turns that diagnostic into a two-part pipeline. A perception model, fine-tuned from Qwen3-VL-4B, predicts which object categories appear in recent images, whether the requested goal is visible, and a distance range. A separate Qwen3-VL-2B policy uses that text with its visual history to choose the next action. The text is supervised from scene annotations during training, so the researchers can train perception and navigation separately.
On the paper’s audited HM3D-OVON unseen split, which excludes episodes affected by annotation errors, StopWatch scored 69.6% success, compared with 60.8% for LightNav-0, the strongest baseline reported in that comparison. The split contains 2,309 of the original 3,000 episodes. The paper also reports the top result among compared methods on six tasks, but those table entries mix official and corrected evaluation protocols.
The benchmark itself changes the score
The authors say object labels in the benchmark use inconsistent synonyms and include missing or incorrect annotations. They merged 2,408 labels into 540 object classes, manually audited episodes, and excluded cases affected by annotation errors. Under this corrected evaluation, scores rose by 13 percentage points for LightNav-0 and 16 points for StopWatch on HM3D-OVON unseen episodes. That matters because the official score can mark a valid stop as a failure when the object has a different label.
Simulation results leave the real-home question open
The paper’s own limitation is central: these tests use simulated, static indoor scenes, not a physical home robot. The training annotations come from the simulator: they identify which object categories are visible and how far the goal is. At test time, the perception model must estimate those details from camera images. Perception remains a bottleneck: the authors attribute 43% of failures in one audited set to incorrectly judging whether the goal was visible, and another 26% to estimating its distance incorrectly. Small or thin objects were harder for the model to detect.
StopWatch therefore offers evidence that “when should I stop?” is separable from “how should I search?” in these benchmarks. It does not show that a deployed robot can reliably retrieve household objects.
Sources
- StopWatch paper, results and limitations (arXiv)
- Illustrative image of a separate home-monitoring robot by Enrique Dorronzoro Zubiete et al., via Wikimedia Commons, under CC BY 4.0; see the 2016 project paper.
