A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation
Not on the map. Why?
Abstract (arXiv)
Navigation is a fundamental capability of autonomous systems, yet most existing approaches rely on highly structured models and strong prior assumptions, limiting their robustness in open and uncertain real-world environments. Vision-and-Language Navigation (VLN) offers a promising direction by enabling robots to integrate natural language understanding with visual perception in a data-driven manner. Although VLN has attracted increasing research attention, systematic methodological taxonomy and real-world validation remain limited. This survey presents a comprehensive review of VLN research. Specifically, state-of-the-art methods are organized along two orthogonal dimensions: action paradigms, including hierarchical and monolithic frameworks, and model paradigms, including discriminative and generative approaches. A critical analysis of their respective strengths and limitations is provided. Additionally, we conduct a systematic real-world evaluation of representative VLN system configurations on a physical robotic platform. Experiments across ten diverse real-world scenes show a substantial performance gap between simulation and real-world deployment under the tested configurations: a representative monolithic RGB-only method achieves 61% success in simulation but drops to 22% in real-world deployment, while a hierarchical framework achieves a higher real-world success rate of 51%, suggesting stronger robustness in our evaluation setting. Finally, we highlight key challenges in perception, decision-making, and control that must be addressed in future research.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 228 of its 281 references that are in the map. First posted on arXiv 9 Jul 2026.
Builds on
228 of 281 references.
References · 228
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
38 more
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
- 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 801
- 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
- 2018On Evaluation of Embodied Navigation Agentscited by 526
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
- 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
- 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
- 2022Visual Language Maps for Robot Navigationcited by 362
- 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
- 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
- 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
- 2017Embodied Question Answeringcited by 335
- 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
- 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
- 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
- 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
- 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
- 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
- 2024Navigation World Modelscited by 262
- 2019Vision-and-Dialog Navigationcited by 246
- 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
- 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
- 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 234
- 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
- 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
- 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
50 most cited of 228 All 228 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.