Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Control
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Social robot navigation (SRN) requires more than geometric path planning; it demands understanding human intentions, social norms, and contextual cues to generate socially compliant behaviors. Although classical navigation methods provide reliable metric planning and collision avoidance, they often lack the semantic reasoning capabilities necessary for operation in complex human-centered environments. Recent advances in Vision-Language Models (VLMs) have opened new opportunities for SRN by enabling high-level VLM understanding, commonsense reasoning, and natural language interaction. However, a fundamental challenge remains: how to integrate VLMs into real-time, safety-critical navigation systems and reliably translate their high-level reasoning into grounded navigation actions. In this survey, we present a unified perspective of VLM-based SRN and organize existing approaches into three interconnected components: high-level VLM reasoning, low-level planning and control, and intermediate mechanisms that bridge reasoning and action. Based on this perspective, we propose a structured roadmap for coupling VLMs with navigation systems, covering semantic reasoning, evaluators, spatial grounding, intermediate representations, and control modules. The roadmap highlights both the strengths of VLMs and the necessity of hybrid architectures for practical deployment. We further review representative datasets and evaluation platforms developed for SRN. Finally, we discuss key open challenges. This survey aims to provide a foundation for building reliable, socially compliant, and deployable VLM-enabled navigation systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 124 references that are in the map. First posted on arXiv 27 Jun 2026.
Builds on
26 of 124 references.
Closest ancestors · 3
Same direction, newest first
- 2025MUSON: A Reasoning-oriented Multimodal Dataset for Socially Compliant Navigation in Urban Environments28 Dec
- 2025ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation12 Mar
- 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigation20 Feb
Other references · 23
- 2023GPT-4 Technical Reportcited by 18,396
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
- 2018On Evaluation of Embodied Navigation Agentscited by 547
- 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
- 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
- 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 194
- 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 165
- 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 165
- 2022Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graphcited by 53
11 more
- 2025OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigationcited by 38
- 2024LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videoscited by 32
- 2023Grounding Classical Task Planners via Vision-Language Modelscited by 23
- 2019JRDB: A Dataset and Benchmark of Egocentric Robot Visual Perception of Humans in Built Environmentscited by 12
- 2022A Protocol for Validating Social Navigation Policiescited by 12
- 2024SELFI: Autonomous Self-Improvement with Reinforcement Learning for Social Navigationcited by 8
- 2023SOCIALGYM 2.0: Simulator for Multi-Agent Social Robot Navigation in Shared Human Spacescited by 5
- 2025SocialNav-MoE: A Mixture-of-Experts Vision Language Model for Socially Compliant Navigation with Reinforcement Fine-Tuningcited by 5
- 2026Walk With Me: Long-Horizon Social Navigation for Human-Centric Outdoor Assistancecited by 3
- 2025A short methodological review on social robot navigation benchmarkingcited by 1
- 2026Probing Prompt Design for Socially Compliant Robot Navigation with Vision Language Modelscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policies
- 2026From Obstacles to Etiquette: Robot Social Navigation with VLM-Informed Path Selection
- 2024VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models
- 2026STARS: From Spatiotemporal Dynamics to Social Representations in Human-Robot Interaction
- 2025SocialNav-SUB: Benchmarking VLMs for Scene Understanding in Social Robot Navigation
- 2026G2-Nav: Grounded and Guarded Vision-Language Costmaps for Robot Social Navigation ↗
- 2024Social-LLaVA: Enhancing Robot Navigation through Human-Language Reasoning in Social Spaces
- 2026SOPD-SocialNav: Selective On-Policy Distillation for Vision-Language Social Navigation