# From Obstacles to Etiquette: Robot Social Navigation with VLM-Informed Path Selection

arXiv: https://arxiv.org/abs/2602.09002 · HTML: https://mlsanity.com/paper/2602.09002/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 43 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 30 of 34)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [YOLOv10: Real-Time End-to-End Object Detection](https://mlsanity.com/paper/2405.14458/) (2024)
- [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023)
- [ViNT: A Foundation Model for Visual Navigation](https://mlsanity.com/paper/2306.14846/) (2023)
- [Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation](https://mlsanity.com/paper/2403.17846/) (2024)
- [PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs](https://mlsanity.com/paper/2402.07872/) (2024)
- [Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning](https://mlsanity.com/paper/1809.08835/) (2018)
- [Socially Aware Motion Planning with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.08862/) (2017)
- [Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation](https://mlsanity.com/paper/2203.15041/) (2022)
- [Core Challenges of Social Robot Navigation: A Survey](https://mlsanity.com/paper/2103.05668/) (2021)
- [SACSoN: Scalable Autonomous Control for Social Navigation](https://mlsanity.com/paper/2306.01874/) (2023)
- [Principles and Guidelines for Evaluating Social Robot Navigation Algorithms](https://mlsanity.com/paper/2306.16740/) (2023)
- [Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graph](https://mlsanity.com/paper/2203.01821/) (2022)
- [VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models](https://mlsanity.com/paper/2404.00210/) (2024)
- [CoNVOI: Context-aware Navigation using Vision Language Models in Outdoor and Indoor Environments](https://mlsanity.com/paper/2403.15637/) (2024)
- [Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning](https://mlsanity.com/paper/2011.04820/) (2020)
- [DenseCAvoid: Real-time Navigation in Dense Crowds using Anticipatory Behaviors](https://mlsanity.com/paper/2002.03038/) (2020)
- [Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation](https://mlsanity.com/paper/2401.12275/) (2024)
- [OLiVia-Nav: An Online Lifelong Vision Language Approach for Mobile Robot Social Navigation](https://mlsanity.com/paper/2409.13675/) (2024)
- [Robotic Guide Dog: Leading a Human with Leash-Guided Hybrid Physical Interaction](https://mlsanity.com/paper/2103.14300/) (2021)
- [Group-based Motion Prediction for Navigation in Crowded Environments](https://mlsanity.com/paper/2107.11637/) (2021)
- [Rethinking Social Robot Navigation: Leveraging the Best of Two Worlds](https://mlsanity.com/paper/2309.13466/) (2023)
- [SICNav: Safe and Interactive Crowd Navigation using Model Predictive Control and Bilevel Optimization](https://mlsanity.com/paper/2310.10982/) (2023)
- [Winding Through: Crowd Navigation via Topological Invariance](https://mlsanity.com/paper/2109.05084/) (2021)
- [VL-TGS: Trajectory Generation and Selection using Vision Language Models in Mapless Outdoor Environments](https://mlsanity.com/paper/2408.02454/) (2024)
- [DR-MPC: Deep Residual Model Predictive Control for Real-world Social Navigation](https://mlsanity.com/paper/2410.10646/) (2024)
- [Interactive Navigation in Environments with Traversable Obstacles Using Large Language and Vision-Language Models](https://mlsanity.com/paper/2310.08873/) (2023)
- [SELFI: Autonomous Self-Improvement with Reinforcement Learning for Social Navigation](https://mlsanity.com/paper/2403.00991/) (2024)

## Built on by (most cited first, 5 of 5)

- [Enhancing Lightweight Vision Language Models through Group Competitive Learning for Socially Compliant Navigation](https://mlsanity.com/paper/2603.11447/) (2026)
- [Act on What You See: Unlocking Safe Social Navigation in Vision-Language-Action Models](https://mlsanity.com/paper/2606.10495/) (2026)
- [Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policies](https://mlsanity.com/paper/2607.10991/) (2026)
- [Commonsense-Grounded Path Planning from Abstract Instructions](https://mlsanity.com/paper/2609.22813/) (2026)
- [Where Should I Join? Robot Group Joining via Language-Guided Goal Prediction](https://mlsanity.com/paper/2609.28467/) (2026)

## Abstract (arXiv)

Navigating socially in human environments requires more than satisfying geometric constraints, as collision-free paths may still interfere with ongoing activities or conflict with social norms. Addressing this challenge calls for analyzing interactions between agents and incorporating common-sense reasoning into planning. This paper presents a social robot navigation framework that integrates geometric planning with contextual social reasoning. The system first extracts obstacles and human dynamics to generate geometrically feasible candidate paths, then leverages a fine-tuned vision-language model (VLM) to evaluate these paths, informed by contextually grounded social expectations, selecting a socially optimized path for the controller. This task-specific VLM distills social reasoning from large foundation models into a smaller and efficient model, allowing the framework to perform real-time adaptation in diverse human-robot interaction contexts. Experiments in four social navigation contexts demonstrate that our method achieves the best overall performance with the lowest personal space violation duration, the minimal pedestrian-facing time, and no social zone intrusions. Project page: https://path-etiquette.github.io