# Principles and Guidelines for Evaluating Social Robot Navigation Algorithms

arXiv: https://arxiv.org/abs/2306.16740 · HTML: https://mlsanity.com/paper/2306.16740/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 185 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 30 of 69)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames](https://mlsanity.com/paper/1911.00357/) (2019)
- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2009.14471/) (2020)
- [iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks](https://mlsanity.com/paper/2108.03272/) (2021)
- [RoboTHOR: An Open Simulation-to-Real Embodied AI Platform](https://mlsanity.com/paper/2004.06799/) (2020)
- [Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning](https://mlsanity.com/paper/1809.08835/) (2018)
- [iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenes](https://mlsanity.com/paper/2012.02924/) (2020)
- [Socially Aware Motion Planning with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.08862/) (2017)
- [Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation](https://mlsanity.com/paper/2203.15041/) (2022)
- [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016)
- [Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1805.01956/) (2018)
- [Behavior Trees in Robotics and AI: An Introduction](https://mlsanity.com/paper/1709.00084/) (2017)
- [Core Challenges of Social Robot Navigation: A Survey](https://mlsanity.com/paper/2103.05668/) (2021)
- [Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.10082/) (2017)
- [Learning Navigation Behaviors End-to-End with AutoRL](https://mlsanity.com/paper/1809.10124/) (2018)
- [Ablation Studies in Artificial Neural Networks](https://mlsanity.com/paper/1901.08644/) (2019)
- [Motion Planning and Control for Mobile Robot Navigation Using Machine Learning: a Survey](https://mlsanity.com/paper/2011.13112/) (2020)
- [Collision Avoidance in Pedestrian-Rich Environments with Deep Reinforcement Learning](https://mlsanity.com/paper/1910.11689/) (2019)
- [Retrospectives on the Embodied AI Workshop](https://mlsanity.com/paper/2210.06849/) (2022)
- [TH"OR: Human-Robot Navigation Data Collection and Accurate Motion Trajectories Dataset](https://mlsanity.com/paper/1909.04403/) (2019)

## Built on by (most cited first, 30 of 56)

- [Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots](https://mlsanity.com/paper/2310.13724/) (2023)
- [Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks](https://mlsanity.com/paper/2412.06224/) (2024)
- [TrackVLA: Embodied Visual Tracking in the Wild](https://mlsanity.com/paper/2505.23189/) (2025)
- [VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models](https://mlsanity.com/paper/2404.00210/) (2024)
- [TrackVLA++: Unleashing Reasoning and Memory Capabilities in VLA Models for Embodied Visual Tracking](https://mlsanity.com/paper/2510.07134/) (2025)
- [EgoNormia: Benchmarking Physical Social Norm Understanding](https://mlsanity.com/paper/2502.20490/) (2025)
- [From Cognition to Precognition: A Future-Aware Framework for Social Navigation](https://mlsanity.com/paper/2409.13244/) (2024)
- [AutoSpatial: Visual-Language Reasoning for Social Robot Navigation through Efficient Spatial Reasoning Learning](https://mlsanity.com/paper/2503.07557/) (2025)
- [SocialNav-SUB: Benchmarking VLMs for Scene Understanding in Social Robot Navigation](https://mlsanity.com/paper/2509.08757/) (2025)
- [SoNIC: Safe Social Navigation with Adaptive Conformal Inference and Constrained Reinforcement Learning](https://mlsanity.com/paper/2407.17460/) (2024)
- [DR-MPC: Deep Residual Model Predictive Control for Real-world Social Navigation](https://mlsanity.com/paper/2410.10646/) (2024)
- [HEIGHT: Heterogeneous Interaction Graph Transformer for Robot Navigation in Crowded and Constrained Environments](https://mlsanity.com/paper/2411.12150/) (2024)
- [PREDILECT: Preferences Delineated with Zero-Shot Language-based Reasoning in Reinforcement Learning](https://mlsanity.com/paper/2402.15420/) (2024)
- [SELFI: Autonomous Self-Improvement with Reinforcement Learning for Social Navigation](https://mlsanity.com/paper/2403.00991/) (2024)
- [Multi-Agent Inverse Reinforcement Learning in Real World Unstructured Pedestrian Crowds](https://mlsanity.com/paper/2405.16439/) (2024)
- [GSON: A Group-based Social Navigation Framework with Large Multimodal Model](https://mlsanity.com/paper/2409.18084/) (2024)
- [Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handling](https://mlsanity.com/paper/2508.05634/) (2025)
- [Beyond Text: Utilizing Vocal Cues to Improve Decision Making in LLMs for Robot Navigation Tasks](https://mlsanity.com/paper/2402.03494/) (2024)
- ["We are at the mercy of others' opinion": Supporting Blind People in Recreational Window Shopping with AI-infused Technology](https://mlsanity.com/paper/2405.06611/) (2024)
- [ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation](https://mlsanity.com/paper/2503.09820/) (2025)
- [SocialNav-MoE: A Mixture-of-Experts Vision Language Model for Socially Compliant Navigation with Reinforcement Fine-Tuning](https://mlsanity.com/paper/2512.14757/) (2025)
- [From Obstacles to Etiquette: Robot Social Navigation with VLM-Informed Path Selection](https://mlsanity.com/paper/2602.09002/) (2026)
- [A Study on Learning Social Robot Navigation with Multimodal Perception](https://mlsanity.com/paper/2309.12568/) (2023)
- [Arena 4.0: A Comprehensive ROS2 Development and Benchmarking Platform for Human-centric Navigation Using Generative-Model-based Environment Generation](https://mlsanity.com/paper/2409.12471/) (2024)
- [Artificial Intelligent Disobedience: Rethinking the Agency of Our Artificial Teammates](https://mlsanity.com/paper/2506.22276/) (2025)
- [ComposableNav: Instruction-Following Navigation in Dynamic Environments via Composable Diffusion](https://mlsanity.com/paper/2509.17941/) (2025)
- [Predicting Human Perceptions of Robot Performance During Navigation Tasks](https://mlsanity.com/paper/2310.11590/) (2023)
- [Disentangling Uncertainty for Safe Social Navigation using Deep Reinforcement Learning](https://mlsanity.com/paper/2409.10655/) (2024)
- [Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Review](https://mlsanity.com/paper/2506.14831/) (2025)
- [Socially aware navigation for mobile robots: a survey on deep reinforcement learning approaches](https://mlsanity.com/paper/2512.00049/) (2025)

## Abstract (arXiv)

A major challenge to deploying robots widely is navigation in human-populated environments, commonly referred to as social robot navigation. While the field of social navigation has advanced tremendously in recent years, the fair evaluation of algorithms that tackle social navigation remains hard because it involves not just robotic agents moving in static environments but also dynamic human agents and their perceptions of the appropriateness of robot behavior. In contrast, clear, repeatable, and accessible benchmarks have accelerated progress in fields like computer vision, natural language processing and traditional robot navigation by enabling researchers to fairly compare algorithms, revealing limitations of existing solutions and illuminating promising new directions. We believe the same approach can benefit social navigation. In this paper, we pave the road towards common, widely accessible, and repeatable benchmarking criteria to evaluate social robot navigation. Our contributions include (a) a definition of a socially navigating robot as one that respects the principles of safety, comfort, legibility, politeness, social competency, agent understanding, proactivity, and responsiveness to context, (b) guidelines for the use of metrics, development of scenarios, benchmarks, datasets, and simulators to evaluate social navigation, and (c) a design of a social navigation metrics framework to make it easier to compare results from different simulators, robots and datasets.