mlsanityData to 7 Oct 2026

GridMM: Grid Memory Map for Vision-and-Language Navigation

arXiv 2307.12907 ↗First posted 24 Jul 2023cs.CV cs.AIcited by 117

Not on the map. Why?

Abstract (arXiv)

Vision-and-language navigation (VLN) enables the agent to navigate to a remote location following the natural language instruction in 3D environments. To represent the previously visited environment, most approaches for VLN implement memory using recurrent states, topological maps, or top-down semantic maps. In contrast to these approaches, we build the top-down egocentric and dynamically growing Grid Memory Map (i.e., GridMM) to structure the visited environment. From a global perspective, historical observations are projected into a unified grid map in a top-down view, which can better represent the spatial relations of the environment. From a local perspective, we further propose an instruction relevance aggregation method to capture fine-grained visual clues in each grid region. Extensive experiments are conducted on both the REVERIE, R2R, SOON datasets in the discrete environments, and the R2R-CE dataset in the continuous environments, showing the superiority of our proposed method.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). Builds on 56 of its 68 references that are in the map. First posted on arXiv 24 Jul 2023.

Builds on

56 of 68 references.

References · 56

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  6. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  9. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  10. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  11. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  12. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
38 more
  1. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  2. 2022Visual Language Maps for Robot Navigationcited by 362
  3. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  4. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  5. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  6. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  7. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  8. 2019Vision-and-Dialog Navigationcited by 246
  9. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  10. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  11. 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
  12. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  13. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  14. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  15. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  16. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
  17. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
  18. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 115
  19. 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
  20. 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
  21. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  22. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  23. 2020Sim-to-Real Transfer for Vision-and-Language Navigationcited by 88
  24. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 87
  25. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  26. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  27. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 80
  28. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 69
  29. 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learningcited by 60
  30. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  31. 2020Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Viewscited by 57
  32. 2022Learning from Unlabeled 3D Environments for Vision-and-Language Navigationcited by 49
  33. 2021SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environmentscited by 48
  34. 2022ADAPT: Vision-Language Navigation with Modality-Aligned Action Promptscited by 45
  35. 2023KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigationcited by 38
  36. 2022Reinforced Structured State-Evolution for Vision-Language Navigationcited by 36
  37. 2021Hierarchical Object-to-Zone Graph for Object Navigationcited by 34
  38. 2022Episodic Memory Question Answeringcited by 32

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 26; 2025: 37; 2026: 52.

117 papers.

Sort:

By area:

  1. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  2. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  3. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
  4. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
  5. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 80
  6. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  7. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  8. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 63
  9. 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 60
  10. 2024Vision-and-Language Navigation via Causal Learningcited by 51
  11. 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 49
  12. 2025Openfly: A comprehensive platform for aerial vision-language navigationcited by 43
38 more
  1. 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environmentscited by 41
  2. 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 34
  3. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigationcited by 30
  4. 2025SmartWay: Enhanced Waypoint Prediction and Backtracking for Zero-Shot Vision-and-Language Navigationcited by 29
  5. 2023Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigationcited by 25
  6. 2025NavBench: Probing Multimodal Large Language Models for Embodied Navigationcited by 24
  7. 2025Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparitiescited by 24
  8. 2026ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigationcited by 23
  9. 2025Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigationcited by 21
  10. 2024Controllable Navigation Instruction Generation with Chain of Thought Promptingcited by 20
  11. 2024Navigation Instruction Generation with BEV Perception and Large Language Modelscited by 20
  12. 2024Vision-Language Navigation with Energy-Based Policycited by 19
  13. 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Taskscited by 18
  14. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 18
  15. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 17
  16. 2025AstraNav-World: World Model for Foresight Control and Consistencycited by 17
  17. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  18. 2025FlexVLN: Flexible Adaptation for Diverse Vision-and-Language Navigation Taskscited by 16
  19. 2025Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigationcited by 16
  20. 2025MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robotscited by 15
  21. 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Predictioncited by 15
  22. 2025SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearingcited by 14
  23. 2025COSMO: Combination of Selective Memorization for Low-cost Vision-and-Language Navigationcited by 12
  24. 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integrationcited by 12
  25. 2024Mind the Error! Detection and Localization of Instruction Errors in Vision-and-Language Navigationcited by 11
  26. 2025Ground-level Viewpoint Vision-and-Language Navigation in Continuous Environmentscited by 10
  27. 2025GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigationcited by 10
  28. 2024Vision-Language Navigation with Embodied Intelligence: A Surveycited by 9
  29. 2025VLN-Zero: Rapid Exploration and Cache-Enabled Neurosymbolic Vision-Language Planning for Zero-Shot Transfer in Robot Navigationcited by 8
  30. 2026SpatialNav: Leveraging Spatial Scene Graphs for Zero-Shot Vision-and-Language Navigationcited by 8
  31. 2025LoopNav: Benchmarking Spatial Consistency in World Modelscited by 7
  32. 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoningcited by 7
  33. 2026Dual-Anchoring: Addressing State Drift in Vision-Language Navigationcited by 7
  34. 2024MAGIC: Meta-Ability Guided Interactive Chain-of-Distillation for Effective-and-Efficient Vision-and-Language Navigationcited by 6
  35. 2024PRET: Planning with Directed Fidelity Trajectory for Vision and Language Navigationcited by 6
  36. 2024MiniVLN: Efficient Vision-and-Language Navigation by Progressive Knowledge Distillationcited by 6
  37. 2025View Invariant Learning for Vision-Language Navigation in Continuous Environmentscited by 6
  38. 2025NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigationcited by 6

50 most cited of 117 All 117 (JSON)