mlsanityData to 14 Oct 2026

Discuss Before Moving: Visual Language Navigation via Multi-expert Discussions

arXiv 2309.11382 ↗First posted 20 Sep 2023cs.RO cs.AI cs.CLcited by 110

On the map: World models → World models for navigation

Abstract (arXiv)

Visual language navigation (VLN) is an embodied task demanding a wide range of skills encompassing understanding, perception, and planning. For such a multifaceted challenge, previous VLN methods totally rely on one model's own thinking to make predictions within one round. However, existing models, even the most advanced large language model GPT4, still struggle with dealing with multiple tasks by single-round self-thinking. In this work, drawing inspiration from the expert consultation meeting, we introduce a novel zero-shot VLN framework. Within this framework, large models possessing distinct abilities are served as domain experts. Our proposed navigation agent, namely DiscussNav, can actively discuss with these experts to collect essential information before moving at every step. These discussions cover critical navigation subtasks like instruction understanding, environment perception, and completion estimation. Through comprehensive experiments, we demonstrate that discussions with domain experts can effectively facilitate navigation by perceiving instruction-relevant information, correcting inadvertent errors, and sifting through in-consistent movement decisions. The performances on the representative VLN task R2R show that our method surpasses the leading zero-shot VLN model by a large margin on all metrics. Additionally, real-robot experiments display the obvious advantages of our method over single-round self-thinking.

Key facts. Cited by 110 papers in the map (data through 14 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 39 of its 50 references that are in the map. First posted on arXiv 20 Sep 2023.

Builds on

39 of 50 references.

References · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  10. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  11. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  12. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
27 more
  1. 2023Large Language Modelscited by 780
  2. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  3. 2022PAL: Program-aided Language Modelscited by 652
  4. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  5. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
  6. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  7. 2022Visual Language Maps for Robot Navigationcited by 365
  8. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  9. 2023Recognize Anything: A Strong Image Tagging Modelcited by 309
  10. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  11. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  12. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  13. 2019Vision-and-Dialog Navigationcited by 250
  14. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
  15. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 231
  16. 2021TEACh: Task-driven Embodied Agents that Chatcited by 197
  17. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  18. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  19. 2021Episodic Transformer for Vision-and-Language Navigationcited by 156
  20. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
  21. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  22. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  23. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70
  24. 2021Less is More: Generating Grounded Navigation Instructions from Landmarkscited by 64
  25. 2021The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigationcited by 61
  26. 2019Just Ask:An Interactive Learning Framework for Vision and Language Navigationcited by 53
  27. 2022Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigationcited by 51

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 26; 2025: 45; 2026: 38.

110 papers.

Sort:

By area:

  1. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  2. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  3. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  4. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
  5. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 118
  6. 2024Large Multimodal Agents: A Surveycited by 105
  7. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
  8. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 86
  9. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 79
  10. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  11. 2024AppAgent v2: Advanced Agent for Flexible Mobile Interactionscited by 68
  12. 2024Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigationcited by 65
38 more
  1. 2025VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuningcited by 63
  2. 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 62
  3. 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 57
  4. 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 50
  5. 2024COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Modelscited by 45
  6. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 44
  7. 2024CityNav: A Large-Scale Dataset for Real-World Aerial Navigationcited by 42
  8. 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 39
  9. 2025Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answeringcited by 30
  10. 2025SmartWay: Enhanced Waypoint Prediction and Backtracking for Zero-Shot Vision-and-Language Navigationcited by 29
  11. 2025Safety Aware Task Planning via Large Language Models in Roboticscited by 24
  12. 2024RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigationcited by 23
  13. 2023ThinkBot: Embodied Instruction Following with Thought Chain Reasoningcited by 19
  14. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 19
  15. 2025FlexVLN: Flexible Adaptation for Diverse Vision-and-Language Navigation Taskscited by 17
  16. 2025Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigationcited by 17
  17. 2025DreamNav: A Trajectory-Based Imaginative Framework for Zero-Shot Vision-and-Language Navigationcited by 17
  18. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  19. 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Predictioncited by 16
  20. 2025Advancing Audio-Visual Navigation Through Multi-Agent Collaboration in 3D Environmentscited by 14
  21. 2025Exploring Embodied Multimodal Large Models: Development, Datasets, and Future Directionscited by 13
  22. 2025ActiveVLN: Towards Active Exploration via Multi-Turn RL in Vision-and-Language Navigationcited by 13
  23. 2025Dynamic Multi-Target Fusion for Efficient Audio-Visual Navigationcited by 13
  24. 2025MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigationcited by 13
  25. 2025SocialNav: Training Human-Inspired Foundation Model for Socially-Aware Embodied Navigationcited by 12
  26. 2024MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agentscited by 11
  27. 2025GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigationcited by 10
  28. 2026AwareVLN: Reasoning with Self-awareness for Vision-Language Navigationcited by 10
  29. 2024Multimodal LLM Guided Exploration and Active Mapping using Fisher Informationcited by 9
  30. 2025SE-VLN: A Self-Evolving Vision-Language Navigation Framework Based on Multimodal Large Language Modelscited by 9
  31. 2026SpatialNav: Leveraging Spatial Scene Graphs for Zero-Shot Vision-and-Language Navigationcited by 9
  32. 2025Unseen from Seen: Rewriting Observation-Instruction Using Foundation Models for Augmenting Vision-Language Navigationcited by 8
  33. 2026HiMemVLN: Enhancing Reliability of Open-Source Zero-Shot Vision-and-Language Navigation with Hierarchical Memory Systemcited by 8
  34. 2024RoboCAS: A Benchmark for Robotic Manipulation in Complex Object Arrangement Scenarioscited by 7
  35. 2024Navigating the Nuances: A Fine-grained Evaluation of Vision-Language Navigationcited by 7
  36. 2025Active Test-time Vision-Language Navigationcited by 7
  37. 2025CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulationcited by 7
  38. 2025Fast-SmartWay: Panoramic-Free End-to-End Zero-Shot Vision-and-Language Navigationcited by 7

50 most cited of 110 All 110 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering
  2. 2024TINA: Think, Interaction, and Action Framework for Zero-Shot Vision Language Navigation
  3. 2025Aux-Think: Exploring Reasoning Strategies for Data-Efficient Vision-Language Navigation
  4. 2026SFCo-Nav: Efficient Zero-Shot Visual Language Navigation via Collaboration of Slow LLM and Fast Attributed Graph Alignment
  5. 2026What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior
  6. 2025Breaking Down and Building Up: Mixture of Skill-Based Vision-and-Language Navigation Agents
  7. 2026Query, Align, and Distill: Navigation-Aware Cross-Modal Interaction for Efficient Vision-and-Language Navigation
  8. 2025Run, Ruminate, and Regulate: A Dual-process Thinking System for Vision-and-Language Navigation