MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation
Not on the map. Why?
Abstract (arXiv)
Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to maintain long-horizon visual history for trajectory consistency while executing actions with low latency. Existing video-based VLN approaches typically struggle to satisfy both demands simultaneously. To address these challenges, we propose MemVLN, a novel VLN framework that achieves state-of-the-art performance with real-time inference efficiency (14 FPS). MemVLN utilizes a visual encoder to process continuous observations and a Large Language Model (LLM) to interpret instructions and generate actions. Central to our approach is an Episodic Memory management that applies pyramidal resolutions. This mechanism concentrates computation on immediate percepts while retaining compressed long-term history. Complementing to this design, we introduce Procedural Memory for fast action with a compact vocabulary of atomic mid-level actions to bypass auto-regressive decoding latency. Experiments on VLN-CE show that MemVLN-4B surpasses the baseline Qwen3-VL-4B architecture by 5.8% SR in R2R and 9.7% SR in RxR, while achieving a 7× speedup in inference latency.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 47 references that are in the map. First posted on arXiv 26 Jul 2026.
Builds on
44 of 47 references.
References · 44
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023GPT-4 Technical Reportcited by 18,132
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Visual Instruction Tuningcited by 7,792
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
- 2023VILA: On Pre-training for Visual Language Modelscited by 686
32 more
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
- 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
- 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
- 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
- 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
- 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
- 2019Vision-and-Dialog Navigationcited by 246
- 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
- 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
- 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
- 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
- 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 146
- 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
- 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
- 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
- 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 115
- 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
- 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
- 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
- 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
- 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
- 2020Sim-to-Real Transfer for Vision-and-Language Navigationcited by 88
- 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 87
- 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
- 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
- 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 80
- 2024DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMscited by 72
- 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 34
- 2024Lyra: An Efficient and Speech-Centric Framework for Omni-Cognitioncited by 20
Built on by
Not cited yet.