mlsanityData to 14 Oct 2026

Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigation

arXiv 2510.08553 ↗First posted 9 Oct 2025cs.CV cs.AI cs.ROcited by 3

On the map: World models → World models for navigation

Abstract (arXiv)

Vision-and-Language Navigation (VLN) requires agents to follow natural language instructions through environments, with memory-persistent variants demanding progressive improvement through accumulated experience. Existing approaches for memory-persistent VLN face critical limitations: they lack effective memory access mechanisms, instead relying on entire memory incorporation or fixed-horizon lookup, and predominantly store only environmental observations while neglecting navigation behavioral patterns that encode valuable decision-making strategies. We present Memoir, which employs imagination as a retrieval mechanism grounded by explicit memory: a world model imagines future navigation states as queries to selectively retrieve relevant environmental observations and behavioral histories. The approach comprises: 1) a language-conditioned world model that imagines future states serving dual purposes: encoding experiences for storage and generating retrieval queries; 2) Hybrid Viewpoint-Level Memory that anchors both observations and behavioral patterns to viewpoints, enabling hybrid retrieval; and 3) an experience-augmented navigation model that integrates retrieved knowledge through specialized encoders. Extensive evaluation across diverse memory-persistent VLN benchmarks with 10 distinct testing scenarios demonstrates Memoir's effectiveness: significant improvements across all scenarios, with 5.4% SPL gains on IR2R over the best memory-persistent baseline, accompanied by 8.3x training speedup and 74% inference memory reduction. The results validate that predictive retrieval of both environmental and behavioral memories enables more effective navigation, with analysis indicating substantial headroom (73.3% vs 93.4% upper bound) for this imagination-guided paradigm.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 48 of its 56 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

48 of 56 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments30 Jun
  2. 2024Navigation World Models4 Dec
  3. 2024OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representation26 Mar
  4. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation14 Aug
  5. 2021Pathdreamer: A World Model for Indoor Navigation18 May

Other references · 43

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  3. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  5. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  6. 2018On Evaluation of Embodied Navigation Agentscited by 547
  7. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 415
  8. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  9. 2022Visual Language Maps for Robot Navigationcited by 365
  10. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  11. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 333
  12. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
31 more
  1. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  2. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  3. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 231
  4. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 196
  5. 2020Neural Topological SLAM for Visual Navigationcited by 195
  6. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  7. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  8. 2020Occupancy Anticipation for Efficient Exploration and Navigationcited by 106
  9. 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
  10. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  11. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  12. 2017Neural Map: Structured Memory for Deep Reinforcement Learningcited by 90
  13. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  14. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 72
  15. 2020Tent: Fully Test-time Adaptation by Entropy Minimizationcited by 69
  16. 2023Learning to Model the World with Languagecited by 68
  17. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstructioncited by 66
  18. 2020Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Viewscited by 58
  19. 2020Active Visual Information Gathering for Vision-Language Navigationcited by 55
  20. 2023Improving Vision-and-Language Navigation by Generating Future-View Image Semanticscited by 46
  21. 2022Topological Semantic Graph Memory for Image-Goal Navigationcited by 45
  22. 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45
  23. 2022Iterative Vision-and-Language Navigationcited by 35
  24. 2024FLAME: Learning to Navigate with Multimodal LLM in Urban Environmentscited by 19
  25. 2023360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye Viewcited by 18
  26. 2025General Scene Adaptation for Vision-and-Language Navigationcited by 17
  27. 2025MonoDream: Monocular Vision-Language Navigation with Panoramic Dreamingcited by 16
  28. 2021Model-Based Episodic Memory Induces Dynamic Hybrid Controlscited by 14
  29. 2023ESceme: Vision-and-Language Navigation with Episodic Scene Memorycited by 10
  30. 2025SE-VLN: A Self-Evolving Vision-Language Navigation Framework Based on Multimodal Large Language Modelscited by 9
  31. 2024Planning from Imagination: Episodic Simulation and Episodic Memory for Vision-and-Language Navigationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026HIMM: Human-Inspired Long-Term Memory Modeling for Embodied Exploration and Question Answeringcited by 3
  2. 2026CLOSER-VLN: Closed-Loop Self-Verified Retrieval-Augmented Reasoning for Aerial Vision-Language Navigationcited by 2
  3. 2026Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Methodcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CMMR-VLN: Vision-and-Language Navigation via Continual Multimodal Memory Retrieval
  2. 2025VISTA: Generative Visual Imagination for Vision-and-Language Navigation
  3. 2026MemoryVLN: From Ego-Aware Memory Representation to Cross-Context Memory Reuse in Vision-Language Navigation
  4. 2026HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigation
  5. 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Prediction
  6. 2025ImagineNav++: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination
  7. 2026Learning to Retrieve Navigable Candidates for Efficient Vision-and-Language Navigation
  8. 2026Thinking Ahead: Prospection-Guided Retrieval of Memory with Language Models