mlsanityData to 7 Oct 2026

MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation

arXiv 2607.23504 ↗First posted 26 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to maintain long-horizon visual history for trajectory consistency while executing actions with low latency. Existing video-based VLN approaches typically struggle to satisfy both demands simultaneously. To address these challenges, we propose MemVLN, a novel VLN framework that achieves state-of-the-art performance with real-time inference efficiency (14 FPS). MemVLN utilizes a visual encoder to process continuous observations and a Large Language Model (LLM) to interpret instructions and generate actions. Central to our approach is an Episodic Memory management that applies pyramidal resolutions. This mechanism concentrates computation on immediate percepts while retaining compressed long-term history. Complementing to this design, we introduce Procedural Memory for fast action with a compact vocabulary of atomic mid-level actions to bypass auto-regressive decoding latency. Experiments on VLN-CE show that MemVLN-4B surpasses the baseline Qwen3-VL-4B architecture by 5.8% SR in R2R and 9.7% SR in RxR, while achieving a 7× speedup in inference latency.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 47 references that are in the map. First posted on arXiv 26 Jul 2026.

Builds on

44 of 47 references.

References · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  9. 2025Qwen3-VL Technical Reportcited by 2,534
  10. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  11. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  12. 2023VILA: On Pre-training for Visual Language Modelscited by 686
32 more
  1. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  2. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
  3. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  4. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  5. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  6. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  7. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  8. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  9. 2019Vision-and-Dialog Navigationcited by 246
  10. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  11. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  12. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  13. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  14. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  15. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 146
  16. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  17. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
  18. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  19. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 115
  20. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  21. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
  22. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
  23. 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
  24. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  25. 2020Sim-to-Real Transfer for Vision-and-Language Navigationcited by 88
  26. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 87
  27. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  28. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  29. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 80
  30. 2024DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMscited by 72
  31. 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 34
  32. 2024Lyra: An Efficient and Speech-Centric Framework for Omni-Cognitioncited by 20

Built on by

Not cited yet.