# Toward Human-Like Social Robot Navigation: A Large-Scale, Multi-Modal, Social Human Navigation Dataset

arXiv: https://arxiv.org/abs/2303.14880 · HTML: https://mlsanity.com/paper/2303.14880/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2023.

## Built on by (most cited first, 30 of 31)

- [SACSoN: Scalable Autonomous Control for Social Navigation](https://mlsanity.com/paper/2306.01874/) (2023)
- [Principles and Guidelines for Evaluating Social Robot Navigation Algorithms](https://mlsanity.com/paper/2306.16740/) (2023)
- [VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models](https://mlsanity.com/paper/2404.00210/) (2024)
- [SANPO: A Scene Understanding, Accessibility and Human Navigation Dataset](https://mlsanity.com/paper/2309.12172/) (2023)
- [SocialNav-SUB: Benchmarking VLMs for Scene Understanding in Social Robot Navigation](https://mlsanity.com/paper/2509.08757/) (2025)
- [LookOut: Real-World Humanoid Egocentric Navigation](https://mlsanity.com/paper/2508.14466/) (2025)
- [Reinforcement Learning for Wheeled Mobility on Vertically Challenging Terrain](https://mlsanity.com/paper/2409.02383/) (2024)
- [GSON: A Group-based Social Navigation Framework with Large Multimodal Model](https://mlsanity.com/paper/2409.18084/) (2024)
- [VANP: Learning Where to See for Navigation with Self-Supervised Vision-Action Pre-Training](https://mlsanity.com/paper/2403.08109/) (2024)
- [SocialNav-MoE: A Mixture-of-Experts Vision Language Model for Socially Compliant Navigation with Reinforcement Fine-Tuning](https://mlsanity.com/paper/2512.14757/) (2025)
- [A Study on Learning Social Robot Navigation with Multimodal Perception](https://mlsanity.com/paper/2309.12568/) (2023)
- [HRIBench: Benchmarking Vision-Language Models for Real-Time Human Perception in Human-Robot Interaction](https://mlsanity.com/paper/2506.20566/) (2025)
- [MUSON: A Reasoning-oriented Multimodal Dataset for Socially Compliant Navigation in Urban Environments](https://mlsanity.com/paper/2512.22867/) (2025)
- [mmWalk: Towards Multi-modal Multi-view Walking Assistance](https://mlsanity.com/paper/2510.11520/) (2025)
- [EgoCogNav: Cognition-aware Human Egocentric Navigation](https://mlsanity.com/paper/2511.17581/) (2025)
- [Ratatouille: Imitation Learning Ingredients for Real-world Social Robot Navigation](https://mlsanity.com/paper/2509.17204/) (2025)
- [Target-Bench: Can Video World Models Achieve Mapless Path Planning with Semantic Targets?](https://mlsanity.com/paper/2511.17792/) (2025)
- [Dyna-LfLH: Learning Agile Navigation in Dynamic Environments from Learned Hallucination](https://mlsanity.com/paper/2403.17231/) (2024)
- [Probing Prompt Design for Socially Compliant Robot Navigation with Vision Language Models](https://mlsanity.com/paper/2601.14622/) (2026)
- [TRANS: Terrain-aware Reinforcement Learning for Agile Navigation of Quadruped Robots under Social Interactions](https://mlsanity.com/paper/2602.12724/) (2026)
- [EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction](https://mlsanity.com/paper/2605.19004/) (2026)
- [Enhancing Indoor Mobility with Connected Sensor Nodes: A Real-Time, Delay-Aware Cooperative Perception Approach](https://mlsanity.com/paper/2411.02624/) (2024)
- [Stereo4DWalker: Learning 4D-aware Embodied Urban Navigation from Internet Stereo Videos](https://mlsanity.com/paper/2512.10956/) (2025)
- [AutoTraces: Autoregressive Trajectory Forecasting via Multimodal Large Language Models](https://mlsanity.com/paper/2603.07989/) (2026)
- [Act on What You See: Unlocking Safe Social Navigation in Vision-Language-Action Models](https://mlsanity.com/paper/2606.10495/) (2026)
- [From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation](https://mlsanity.com/paper/2606.12603/) (2026)
- [HumAIN: Human-Aware Implicit Social Robot Navigation](https://mlsanity.com/paper/2607.07357/) (2026)
- [ACME: A Multi-Cultural, Multi-Embodiment Social-Navigation Dataset](https://mlsanity.com/paper/2607.21964/) (2026)
- [Failing Gracefully: Mitigating Impact of Inevitable Robot Failures](https://mlsanity.com/paper/2608.05313/) (2026)
- [RECAST: Recasting Vision-Language Semantics into an Actionable Cost Map for Robot Navigation](https://mlsanity.com/paper/2609.32595/) (2026)

## Abstract (arXiv)

Humans are well-adept at navigating public spaces shared with others, where current autonomous mobile robots still struggle: while safely and efficiently reaching their goals, humans communicate their intentions and conform to unwritten social norms on a daily basis; conversely, robots become clumsy in those daily social scenarios, getting stuck in dense crowds, surprising nearby pedestrians, or even causing collisions. While recent research on robot learning has shown promises in data-driven social robot navigation, good-quality training data is still difficult to acquire through either trial and error or expert demonstrations. In this work, we propose to utilize the body of rich, widely available, social human navigation data in many natural human-inhabited public spaces for robots to learn similar, human-like, socially compliant navigation behaviors. To be specific, we design an open-source egocentric data collection sensor suite wearable by walking humans to provide multi-modal robot perception data; we collect a large-scale (~100 km, 20 hours, 300 trials, 13 humans) dataset in a variety of public spaces which contain numerous natural social navigation interactions; we analyze our dataset, demonstrate its usability, and point out future research directions and use cases.