mlsanityData to 7 Oct 2026

Vision-Language Navigation with Embodied Intelligence: A Survey

arXiv 2402.14304 ↗First posted 22 Feb 2024cs.RO cs.AI cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

As a long-term vision in the field of artificial intelligence, the core goal of embodied intelligence is to improve the perception, understanding, and interaction capabilities of agents and the environment. Vision-language navigation (VLN), as a critical research path to achieve embodied intelligence, focuses on exploring how agents use natural language to communicate effectively with humans, receive and understand instructions, and ultimately rely on visual information to achieve accurate navigation. VLN integrates artificial intelligence, natural language processing, computer vision, and robotics. This field faces technical challenges but shows potential for application such as human-computer interaction. However, due to the complex process involved from language understanding to action execution, VLN faces the problem of aligning visual information and language instructions, improving generalization ability, and many other challenges. This survey systematically reviews the research progress of VLN and details the research direction of VLN with embodied intelligence. After a detailed summary of its system architecture and research based on methods and commonly used benchmark datasets, we comprehensively analyze the problems and challenges faced by current research and explore the future development direction of this field, aiming to provide a practical reference for researchers.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 132 of its 176 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

132 of 176 references.

References · 132

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2017Mask R-CNNcited by 7,259
  7. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  8. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  11. 2019Towards VQA Models That Can Readcited by 1,696
  12. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
38 more
  1. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  3. 2016YOLO9000: Better, Faster, Strongercited by 1,429
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  5. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  6. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  7. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  8. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  9. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  10. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  11. 2014Distributed Representations of Sentences and Documentscited by 677
  12. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
  13. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  14. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  15. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 476
  16. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  17. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  18. 2022Visual Language Maps for Robot Navigationcited by 362
  19. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  20. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  21. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  22. 2017Embodied Question Answeringcited by 335
  23. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  24. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  25. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  26. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  27. 2019Vision-and-Dialog Navigationcited by 246
  28. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  29. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  30. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  31. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  32. 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
  33. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 177
  34. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  35. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  36. 2019Talk2Car: Taking Control of Your Self-Driving Carcited by 141
  37. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 129
  38. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129

50 most cited of 132 All 132 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 2.

9 papers.

  1. 2025CityNavAgent: Aerial Vision-and-Language Navigation with Hierarchical Semantic Planning and Global Memorycited by 30
  2. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  3. 2025NaviMaster: Learning a Unified Policy for GUI and Embodied Navigation Taskscited by 10
  4. 2025CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?cited by 8
  5. 2025Active Test-time Vision-Language Navigationcited by 7
  6. 2025HA-VLN 2.0: An Open Benchmark and Leaderboard for Human-Aware Navigation in Discrete and Continuous Environments with Dynamic Multi-Human Interactionscited by 6
  7. 2025Endowing Embodied Agents with Spatial Reasoning Capabilities for Vision-and-Language Navigationcited by 4
  8. 2026GroundControl: Anticipating Navigation Failures in Vision-Language Agents via Trajectory-Consistent Uncertainty Estimatescited by 2
  9. 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1