mlsanityData to 7 Oct 2026

HA-VLN 2.0: An Open Benchmark and Leaderboard for Human-Aware Navigation in Discrete and Continuous Environments with Dynamic Multi-Human Interactions

arXiv 2503.14229 ↗First posted 18 Mar 2025cs.AI cs.CV cs.ROcited by 6

Not on the map. Why?

Abstract (arXiv)

Vision-and-Language Navigation (VLN) has been studied mainly in either discrete or continuous spaces, with little attention to dynamic, crowded environments. We present HA-VLN 2.0, a unified benchmark introducing explicit social-awareness constraints. Our contributions are: (i) a standardized task and metrics capturing both goal accuracy and personal-space adherence; (ii) HAPS 2.0 dataset and simulators modeling multi-human interactions, outdoor contexts, and finer language-motion alignment; (iii) benchmarks on 16,844 socially grounded instructions, revealing sharp performance drops of leading agents under human dynamics and partial observability; and (iv) real-world robot experiments validating sim-to-real transfer, with an open leaderboard enabling transparent comparison. Results show that explicit social modeling improves navigation robustness and reduces collisions, underscoring necessity of human-centric approaches. By releasing datasets, simulators, baselines, and protocols, HA-VLN 2.0 provides a strong foundation for safe, human-aware navigation research.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 57 of its 84 references that are in the map. First posted on arXiv 18 Mar 2025.

Builds on

57 of 84 references.

References · 57

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  4. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  5. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  6. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  7. 2022Human Motion Diffusion Modelcited by 1,062
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  9. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  10. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
  11. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  12. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
38 more
  1. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  2. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  3. 2017Embodied Question Answeringcited by 335
  4. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  5. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  6. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  7. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  8. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  9. 2019Vision-and-Dialog Navigationcited by 246
  10. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  11. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  12. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  13. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  14. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  15. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  16. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 178
  17. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 177
  18. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  19. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  20. 2017IQA: Visual Question Answering in Interactive Environmentscited by 154
  21. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 129
  22. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  23. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
  24. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  25. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  26. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  27. 2018Building Generalizable Agents with a Realistic and Rich 3D Environmentcited by 109
  28. 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
  29. 2020Sim-to-Real Transfer for Vision-and-Language Navigationcited by 88
  30. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  31. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  32. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  33. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 77
  34. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 66
  35. 2022DialFRED: Dialogue-Enabled Agents for Embodied Instruction Followingcited by 63
  36. 2023Learning Vision-and-Language Navigation from YouTube Videoscited by 47
  37. 2018Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Predictioncited by 35
  38. 2021Vision-Language Navigation: A Survey and Taxonomycited by 33

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresightcited by 11
  2. 2026Language-Conditioned World Modeling for Visual Navigationcited by 9
  3. 2025GoViG: Goal-Conditioned Visual Navigation Instruction Generation via Multimodal Reasoningcited by 6
  4. 2026Rule-VLN: Bridging Perception and Compliance via Semantic Reasoning and Geometric Rectificationcited by 0
  5. 2026HCSG: Human-Centric Semantic-Geometric Reasoning for Vision-Language Navigationcited by 0
  6. 2026MESSENGER: Memory-Enhanced Sequential Scene Flow Estimation via Autoregressive Next-Frame Forecastingcited by 0