mlsanityData to 7 Oct 2026

A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation

arXiv 2607.09792 ↗First posted 9 Jul 2026cs.RO cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Navigation is a fundamental capability of autonomous systems, yet most existing approaches rely on highly structured models and strong prior assumptions, limiting their robustness in open and uncertain real-world environments. Vision-and-Language Navigation (VLN) offers a promising direction by enabling robots to integrate natural language understanding with visual perception in a data-driven manner. Although VLN has attracted increasing research attention, systematic methodological taxonomy and real-world validation remain limited. This survey presents a comprehensive review of VLN research. Specifically, state-of-the-art methods are organized along two orthogonal dimensions: action paradigms, including hierarchical and monolithic frameworks, and model paradigms, including discriminative and generative approaches. A critical analysis of their respective strengths and limitations is provided. Additionally, we conduct a systematic real-world evaluation of representative VLN system configurations on a physical robotic platform. Experiments across ten diverse real-world scenes show a substantial performance gap between simulation and real-world deployment under the tested configurations: a representative monolithic RGB-only method achieves 61% success in simulation but drops to 22% in real-world deployment, while a hierarchical framework achieves a higher real-world success rate of 51%, suggesting stronger robustness in our evaluation setting. Finally, we highlight key challenges in perception, decision-making, and control that must be addressed in future research.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 228 of its 281 references that are in the map. First posted on arXiv 9 Jul 2026.

Builds on

228 of 281 references.

References · 228

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  9. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  11. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  12. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
38 more
  1. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  2. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  3. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  5. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  6. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 801
  7. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  8. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  9. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  10. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  11. 2018On Evaluation of Embodied Navigation Agentscited by 526
  12. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  13. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  14. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  15. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  16. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  17. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  18. 2022Visual Language Maps for Robot Navigationcited by 362
  19. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  20. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  21. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  22. 2017Embodied Question Answeringcited by 335
  23. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  24. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  25. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  26. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  27. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  28. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  29. 2024Navigation World Modelscited by 262
  30. 2019Vision-and-Dialog Navigationcited by 246
  31. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  32. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  33. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 234
  34. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  35. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  36. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  37. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
  38. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201

50 most cited of 228 All 228 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.