# SACSoN: Scalable Autonomous Control for Social Navigation

arXiv: https://arxiv.org/abs/2306.01874 · HTML: https://mlsanity.com/paper/2306.01874/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 55 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 29 of 29)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Semi-parametric Topological Memory for Navigation](https://mlsanity.com/paper/1803.00653/) (2018)
- [Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning](https://mlsanity.com/paper/1809.08835/) (2018)
- [GNM: A General Navigation Model to Drive Any Robot](https://mlsanity.com/paper/2210.03370/) (2022)
- [Socially Aware Motion Planning with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.08862/) (2017)
- [Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation](https://mlsanity.com/paper/2203.15041/) (2022)
- [Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills](https://mlsanity.com/paper/2104.07749/) (2021)
- [Rapid Exploration for Open-World Navigation with Latent Goal Models](https://mlsanity.com/paper/2104.05859/) (2021)
- [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016)
- [Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1805.01956/) (2018)
- [Core Challenges of Social Robot Navigation: A Survey](https://mlsanity.com/paper/2103.05668/) (2021)
- [ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints](https://mlsanity.com/paper/2202.11271/) (2022)
- [Deep Visual MPC-Policy Learning for Navigation](https://mlsanity.com/paper/1903.02749/) (2019)
- [Topological Semantic Graph Memory for Image-Goal Navigation](https://mlsanity.com/paper/2209.08274/) (2022)
- [Scaling Local Control to Large-Scale Topological Navigation](https://mlsanity.com/paper/1909.12329/) (2019)
- [TH"OR: Human-Robot Navigation Data Collection and Accurate Motion Trajectories Dataset](https://mlsanity.com/paper/1909.04403/) (2019)
- [Toward Human-Like Social Robot Navigation: A Large-Scale, Multi-Modal, Social Human Navigation Dataset](https://mlsanity.com/paper/2303.14880/) (2023)
- [Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation](https://mlsanity.com/paper/2209.10780/) (2022)
- [ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentation](https://mlsanity.com/paper/2210.07450/) (2022)
- [A Scalable Framework For Real-Time Multi-Robot, Multi-Human Collision Avoidance](https://mlsanity.com/paper/1811.05929/) (2018)
- [Spatio-Temporal Graph Localization Networks for Image-based Navigation](https://mlsanity.com/paper/2204.13237/) (2022)
- [Occlusion-Aware Crowd Navigation Using People as Sensors](https://mlsanity.com/paper/2210.00552/) (2022)
- [An Approach to Deploy Interactive Robotic Simulators on the Web for HRI Experiments: Results in Social Robot Navigation](https://mlsanity.com/paper/2012.12336/) (2020)
- [Robot Perception of Static and Dynamic Objects with an Autonomous Floor Scrubber](https://mlsanity.com/paper/2002.10158/) (2020)
- [EWareNet: Emotion Aware Human Intent Prediction and Adaptive Spatial Profile Fusion for Social Robot Navigation](https://mlsanity.com/paper/2011.09438/) (2020)
- [Metrics for Evaluating Social Conformity of Crowd Navigation Algorithms](https://mlsanity.com/paper/2202.01045/) (2022)
- [SG-LSTM: Social Group LSTM for Robot Navigation Through Dense Crowds](https://mlsanity.com/paper/2303.04320/) (2023)

## Built on by (most cited first, 30 of 61)

- [Navigation World Models](https://mlsanity.com/paper/2412.03572/) (2024)
- [ViNT: A Foundation Model for Visual Navigation](https://mlsanity.com/paper/2306.14846/) (2023)
- [Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications](https://mlsanity.com/paper/2510.07077/) (2025)
- [NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance](https://mlsanity.com/paper/2505.08712/) (2025)
- [VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models](https://mlsanity.com/paper/2404.00210/) (2024)
- [CityWalker: Learning Embodied Urban Navigation from Web-Scale Videos](https://mlsanity.com/paper/2411.17820/) (2024)
- [OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigation](https://mlsanity.com/paper/2509.19480/) (2025)
- [NaviDiffusor: Cost-Guided Diffusion Model for Visual Navigation](https://mlsanity.com/paper/2504.10003/) (2025)
- [Prior Does Matter: Visual Navigation via Denoising Diffusion Bridge Models](https://mlsanity.com/paper/2504.10041/) (2025)
- [ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigation](https://mlsanity.com/paper/2602.11598/) (2026)
- [A Survey of Interactive Generative Video](https://mlsanity.com/paper/2504.21853/) (2025)
- [XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representations](https://mlsanity.com/paper/2511.02776/) (2025)
- [Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight](https://mlsanity.com/paper/2510.08713/) (2025)
- [Learning to Drive Anywhere with Model-Based Reannotation](https://mlsanity.com/paper/2505.05592/) (2025)
- [FlowNav: Combining Flow Matching and Depth Priors for Efficient Navigation](https://mlsanity.com/paper/2411.09524/) (2024)
- [SocialNav: Training Human-Inspired Foundation Model for Socially-Aware Embodied Navigation](https://mlsanity.com/paper/2511.21135/) (2025)
- [PlaceNav: Topological Navigation through Place Recognition](https://mlsanity.com/paper/2309.17260/) (2023)
- [SocialNav-SUB: Benchmarking VLMs for Scene Understanding in Social Robot Navigation](https://mlsanity.com/paper/2509.08757/) (2025)
- [Sparse Video Generation Propels Real-World Beyond-the-View Vision-Language Navigation](https://mlsanity.com/paper/2602.05827/) (2026)
- [AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge](https://mlsanity.com/paper/2602.13476/) (2026)
- [Language-Conditioned World Modeling for Visual Navigation](https://mlsanity.com/paper/2603.26741/) (2026)
- [HEIGHT: Heterogeneous Interaction Graph Transformer for Robot Navigation in Crowded and Constrained Environments](https://mlsanity.com/paper/2411.12150/) (2024)
- [LookOut: Real-World Humanoid Egocentric Navigation](https://mlsanity.com/paper/2508.14466/) (2025)
- [RAE-NWM: Navigation World Model in Dense Visual Representation Space](https://mlsanity.com/paper/2603.09241/) (2026)
- [SELFI: Autonomous Self-Improvement with Reinforcement Learning for Social Navigation](https://mlsanity.com/paper/2403.00991/) (2024)
- [CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?](https://mlsanity.com/paper/2512.16755/) (2025)
- [NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation](https://mlsanity.com/paper/2606.13494/) (2026)
- [Exploring 3D Human Pose Estimation and Forecasting from the Robot's Perspective: The HARPER Dataset](https://mlsanity.com/paper/2403.14447/) (2024)
- [GSON: A Group-based Social Navigation Framework with Large Multimodal Model](https://mlsanity.com/paper/2409.18084/) (2024)
- [UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectories](https://mlsanity.com/paper/2512.09607/) (2025)

## Abstract (arXiv)

Machine learning provides a powerful tool for building socially compliant robotic systems that go beyond simple predictive models of human behavior. By observing and understanding human interactions from past experiences, learning can enable effective social navigation behaviors directly from data. In this paper, our goal is to develop methods for training policies for socially unobtrusive navigation, such that robots can navigate among humans in ways that don't disturb human behavior. We introduce a definition for such behavior based on the counterfactual perturbation of the human: if the robot had not intruded into the space, would the human have acted in the same way? By minimizing this counterfactual perturbation, we can induce robots to behave in ways that do not alter the natural behavior of humans in the shared space. Instantiating this principle requires training policies to minimize their effect on human behavior, and this in turn requires data that allows us to model the behavior of humans in the presence of robots. Therefore, our approach is based on two key contributions. First, we collect a large dataset where an indoor mobile robot interacts with human bystanders. Second, we utilize this dataset to train policies that minimize counterfactual perturbation. We provide supplementary videos and make publicly available the largest-of-its-kind visual navigation dataset on our project page.