mlsanityData to 7 Oct 2026

Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments

arXiv 2004.02857 ↗First posted 6 Apr 2020cs.CV cs.CL cs.ROcited by 429

Not on the map. Why?

Cited by 35 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We develop a language-guided navigation task set in a continuous 3D environment where agents must execute low-level actions to follow natural language navigation directions. By being situated in continuous environments, this setting lifts a number of assumptions implicit in prior work that represents environments as a sparse graph of panoramas with edges corresponding to navigability. Specifically, our setting drops the presumptions of known environment topologies, short-range oracle navigation, and perfect agent localization. To contextualize this new task, we develop models that mirror many of the advances made in prior settings as well as single-modality baselines. While some of these techniques transfer, we find significantly lower absolute performance in the continuous setting -- suggesting that performance in prior `navigation-graph' settings may be inflated by the strong implicit assumptions.

Key facts. Cited by 429 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 25; 2022: 44; 2023: 46; 2024: 60; 2025: 102; 2026: 151.

429 papers.

Sort:

By area:

  1. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  2. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  3. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
  4. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  5. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  6. 2022Visual Language Maps for Robot Navigationcited by 362
  7. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  8. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  9. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  10. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  11. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  12. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
38 more
  1. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  2. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  3. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  4. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  5. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 170
  6. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  7. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  8. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  9. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
  10. 2020Language-Conditioned Imitation Learning for Robot Manipulation Taskscited by 136
  11. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  12. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  13. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  14. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  15. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
  16. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  17. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 117
  18. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  19. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  20. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 117
  21. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  22. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
  23. 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AIcited by 116
  24. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  25. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
  26. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
  27. 2023HomeRobot: Open-Vocabulary Mobile Manipulationcited by 104
  28. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  29. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 92
  30. 2021A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Executioncited by 91
  31. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  32. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  33. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 87
  34. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  35. 2024MileBench: Benchmarking MLLMs in Long Contextcited by 86
  36. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  37. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 80
  38. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 80

50 most cited of 429 All 429 (JSON)