mlsanityData to 14 Oct 2026

Vision-and-Dialog Navigation

arXiv 1907.04957 ↗First posted 10 Jul 2019cs.CL cs.AI cs.CVcited by 250

Not on the map. Why?

Abstract (arXiv)

Robots navigating in human environments should use language to ask for assistance and be able to understand human responses. To study this challenge, we introduce Cooperative Vision-and-Dialog Navigation, a dataset of over 2k embodied, human-human dialogs situated in simulated, photorealistic home environments. The Navigator asks questions to their partner, the Oracle, who has privileged access to the best next steps the Navigator should take according to a shortest path planner. To train agents that search an environment for a goal location, we define the Navigation from Dialog History task. An agent, given a target object and a dialog history between humans cooperating to find that object, must infer navigation actions towards the goal in unexplored environments. We establish an initial, multi-modal sequence-to-sequence model and demonstrate that looking farther back in the dialog history improves performance. Sourcecode and a live interface demo can be found at https://cvdn.dev/

Key facts. Cited by 250 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 2; 2021: 39; 2022: 44; 2023: 49; 2024: 46; 2025: 45; 2026: 24.

250 papers.

Sort:

By area:

  1. 2023Cognitive Architectures for Language Agentscited by 429
  2. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  3. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  4. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  5. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 237
  6. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  7. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
  8. 2021TEACh: Task-driven Embodied Agents that Chatcited by 197
  9. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  10. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  11. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 166
  12. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
38 more
  1. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
  2. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
  3. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  4. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
  5. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 119
  6. 2021Pathdreamer: A World Model for Indoor Navigationcited by 118
  7. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
  8. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 117
  9. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  10. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
  11. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  12. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
  13. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  14. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  15. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  16. 2023Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigationcited by 74
  17. 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphscited by 73
  18. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 70
  19. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 69
  20. 2023Learning to Model the World with Languagecited by 68
  21. 2024ReMEmbR: Building and Reasoning Over Long-Horizon Spatio-Temporal Memory for Robot Navigationcited by 67
  22. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 67
  23. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 65
  24. 2022DialFRED: Dialogue-Enabled Agents for Embodied Instruction Followingcited by 63
  25. 2021The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigationcited by 61
  26. 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learningcited by 60
  27. 2023"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomycited by 60
  28. 2024GRUtopia: Dream General Robots in a City at Scalecited by 59
  29. 2021Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completioncited by 58
  30. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  31. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
  32. 2022Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigationcited by 51
  33. 2024Vision-and-Language Navigation via Causal Learningcited by 51
  34. 2021Neighbor-view Enhanced Model for Vision and Language Navigationcited by 50
  35. 2022Learning from Unlabeled 3D Environments for Vision-and-Language Navigationcited by 50
  36. 2022Aerial Vision-and-Dialog Navigationcited by 49
  37. 2023LangNav: Language as a Perceptual Representation for Navigationcited by 47
  38. 2022Retrospectives on the Embodied AI Workshopcited by 46

50 most cited of 250 All 250 (JSON)