mlsanityData to 7 Oct 2026

Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models

arXiv 2407.07035 ↗First posted 9 Jul 2024cs.CL cs.CVcited by 79

Not on the map. Why?

Abstract (arXiv)

Vision-and-Language Navigation (VLN) has gained increasing attention over recent years and many approaches have emerged to advance their development. The remarkable achievements of foundation models have shaped the challenges and proposed methods for VLN research. In this survey, we provide a top-down review that adopts a principled framework for embodied planning and reasoning, and emphasizes the current methods and future opportunities leveraging foundation models to address VLN challenges. We hope our in-depth discussions could provide valuable resources and insights: on one hand, to milestone the progress and explore opportunities and potential roles for foundation models in this field, and on the other, to organize different challenges and solutions in VLN to foundation model researchers.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Builds on 163 of its 220 references that are in the map. First posted on arXiv 9 Jul 2024.

Builds on

163 of 220 references.

References · 163

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2015VQA: Visual Question Answeringcited by 2,489
  5. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  7. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  8. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  9. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  10. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  11. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  12. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
38 more
  1. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  2. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  3. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  4. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  5. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  6. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  7. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  8. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
  9. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  10. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  11. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  12. 2022Visual Language Maps for Robot Navigationcited by 362
  13. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
  14. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  15. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  16. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  17. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  18. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  19. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  20. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 302
  21. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  22. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  23. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
  24. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  25. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  26. 2019Visual Entailment: A Novel Task for Fine-Grained Image Understandingcited by 250
  27. 2019Vision-and-Dialog Navigationcited by 246
  28. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  29. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  30. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  31. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 231
  32. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  33. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  34. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  35. 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
  36. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
  37. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  38. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 177

50 most cited of 163 All 163 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 35; 2026: 31.

79 papers.

Sort:

By area:

  1. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  2. 2025TrackVLA: Embodied Visual Tracking in the Wildcited by 46
  3. 2024On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detectioncited by 40
  4. 2025Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detectorcited by 38
  5. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
  6. 2025SmartWay: Enhanced Waypoint Prediction and Backtracking for Zero-Shot Vision-and-Language Navigationcited by 29
  7. 2024SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Modelscited by 26
  8. 2025IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstructioncited by 26
  9. 2025NavBench: Probing Multimodal Large Language Models for Embodied Navigationcited by 24
  10. 2025Embodied AI: From LLMs to World Modelscited by 23
  11. 2024MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Datacited by 21
  12. 2024Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigationcited by 18
38 more
  1. 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Taskscited by 18
  2. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 18
  3. 2024Social-LLaVA: Enhancing Robot Navigation through Human-Language Reasoning in Social Spacescited by 17
  4. 2024Language-guided Hierarchical Fine-grained Image Forgery Detection and Localizationcited by 16
  5. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  6. 2025FlexVLN: Flexible Adaptation for Diverse Vision-and-Language Navigation Taskscited by 16
  7. 2024NeSyCoCo: A Neuro-Symbolic Concept Composer for Compositional Generalizationcited by 13
  8. 2025MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigationcited by 13
  9. 2025COSMO: Combination of Selective Memorization for Low-cost Vision-and-Language Navigationcited by 12
  10. 2026When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoningcited by 12
  11. 2026EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understandingcited by 11
  12. 2025Ground-level Viewpoint Vision-and-Language Navigation in Continuous Environmentscited by 10
  13. 2025Neural Brain: A Neuroscience-inspired Framework for Embodied Agentscited by 10
  14. 2025Safety of Embodied Navigation: A Surveycited by 9
  15. 2025CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?cited by 8
  16. 2025FlySearch: Exploring how vision-language models explorecited by 7
  17. 2025CogDDN: A Cognitive Demand-Driven Navigation with Decision Optimization and Dual-Process Thinkingcited by 7
  18. 2025HA-VLN 2.0: An Open Benchmark and Leaderboard for Human-Aware Navigation in Discrete and Continuous Environments with Dynamic Multi-Human Interactionscited by 6
  19. 2025MEXA: Towards General Multimodal Reasoning with Dynamic Multi-Expert Aggregationcited by 6
  20. 2025GoViG: Goal-Conditioned Visual Navigation Instruction Generation via Multimodal Reasoningcited by 6
  21. 2025Fast-SmartWay: Panoramic-Free End-to-End Zero-Shot Vision-and-Language Navigationcited by 6
  22. 2024Narrowing the Gap between Vision and Action in Navigationcited by 5
  23. 2026On the Holistic Approach for Detecting Human Image Forgerycited by 5
  24. 2026APEX: A Decoupled Memory-based Explorer for Asynchronous Aerial Object Goal Navigationcited by 5
  25. 2025History-Augmented Vision-Language Models for Frontier-Based Zero-Shot Object Navigationcited by 4
  26. 2026Instruction-as-State: Environment-Guided and State-Conditioned Semantic Understanding for Embodied Navigationcited by 4
  27. 2025ComposableNav: Instruction-Following Navigation in Dynamic Environments via Composable Diffusioncited by 3
  28. 2025Vision-and-Language Navigation with Analogical Textual Descriptions in LLMscited by 3
  29. 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
  30. 2025UNeMo: Collaborative Visual-Language Reasoning and Navigation via a Multimodal World Modelcited by 3
  31. 2026OpenFrontier: General Navigation with Visual-Language Grounded Frontierscited by 3
  32. 2025Language and Planning in Robotic Navigation: A Multilingual Evaluation of State-of-the-Art Modelscited by 2
  33. 2025AutoDrive-QA: A Multiple-Choice Benchmark for Vision-Language Evaluation in Urban Autonomous Drivingcited by 2
  34. 2025Prune-Then-Plan: Step-Level Calibration for Stable Frontier Exploration in Embodied Question Answeringcited by 2
  35. 2026ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigationcited by 2
  36. 2026SpaAct: Spatially-Activated Transition Learning with Curriculum Adaptation for Vision-Language Navigationcited by 2
  37. 2026Seeing Isn't Knowing: Do VLMs Know When Not to Answer Spatial Questions (and Why)?cited by 2
  38. 2026LIME: Learning Intent-aware Camera Motion from Egocentric Videocited by 2

50 most cited of 79 All 79 (JSON)