A National University of Singapore team has built a robot navigation system that turns a text description of a human group and a color-and-depth view of a scene into a likely place and direction to join. Tested on Boston Dynamics’ Spot, it reached a valid position in 18 of 20 controlled trials across conversations and queues.
The system finds a group before choosing where to stand
Most social-navigation systems start with a destination. Group joining is different: a request such as joining a queue at a coffee stall leaves the robot to identify which people form the target group and where a newcomer can stand without disrupting it.
The paper, submitted to arXiv on Sept. 23, describes two linked stages. First, the robot detects people in an RGB-D observation, which pairs a color image with depth. Recursive spectral partitioning splits a spatial graph of those people into candidate groups. A model then ranks those candidates against the text instruction using image appearance and person-to-person geometry.
A second model uses the selected group’s positions and body orientations to predict a map of possible robot positions and headings. It draws on formation patterns for queues, face-to-face conversations and audiences facing a focal point. Multiple possible poses matter because there can be more than one socially acceptable place to join.
Offline scores test pose prediction; Spot trials test navigation
Offline tests covered 42 scenes: 15 conversations, 15 queues and 12 audience interactions. The researchers collected 11 audience scenes themselves because that formation is scarce in existing datasets. For the complete system, 89.76% of its sampled poses met the paper’s validity criteria, at about 1.1 seconds per query.
The paper defined valid positions as 0.5 to 1.2 meters from the appropriate group reference, such as the end of a queue, and an orientation within 30 degrees of the relevant direction. For audiences, the robot also had to stay behind the first row without blocking viewers.
For physical tests, the team ran five trials each for seated conversations, standing conversations, static queues and queues that changed as a person joined. Spot reached a valid target in 18 of 20 attempts. The authors attribute the two failures to person-detection and control errors. On Spot’s onboard NVIDIA Jetson Orin computer, full inference took less than 0.7 seconds in the tested medium-density scenes with fewer than 10 people.
Group spacing and scene layout still limit the system
The authors found that distance-based grouping can exclude a genuine member who stands farther from others than expected. In audience scenes, the pose predictor sometimes favored the outermost arc even when closer positions were available. The current system also treats the surroundings mainly as obstacles to avoid, rather than using features such as furniture to suggest a natural place to join.
The study does not establish performance in denser, less structured public spaces.
Sources
Fang et al., “Where Should I Join? Robot Group Joining via Language-Guided Goal Prediction” (arXiv HTML) · PDF
Illustrative image: Web Summit, photographed by Lukas Schulze/Sportsfile, via Wikimedia Commons, CC BY 4.0. The image is not from the research study; no changes made.
.jpg){kind=link}