mlsanityData to 14 Oct 2026

VTM-Nav: Harnessing Cross-Episode Experience for Object-Goal Navigation with Hierarchical Visual-Topological Memory

arXiv 2607.14514 ↗First posted 16 Jul 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Training-free ObjectNav agents increasingly use vision-language models (VLMs), yet typically discard acquired scene knowledge after each request. We study cross-episode ObjectNav, where each request is an independently initialized, single-goal episode and only self-acquired, scene-scoped memory persists across episodes. We ask whether an agent with fixed model parameters and navigation components can reuse such experience without retraining or oracle information. We introduce method, a training-free framework with a persistent Hierarchical Visual-Topological Memory (VTM). VTM uses a coarse room topology to index room-owned visual memories, distinguishes in-room from remote-visible evidence, and retains successful approach cues. For each request, VTM-Nav re-localizes the agent in accumulated scene structure, retrieves target-relevant records from plausible rooms, and grounds memory guidance in candidates derived from the current observation. A conservative execution guard further handles local failures. Under matched 40-step comparisons, VTM-Nav exceeds the memory-reset WMNav control by 4.6, 2.0, and 0.8 SR points on HM3D v0.1, HM3D v0.2, and MP3D, respectively, with comparable or higher SPL. On HM3D, it also exceeds WMNav harnessed by textual memory by 3.1 and 5.5 SR points. These results demonstrate effective reuse of cross-episode scene experience through hierarchical visual-topological memory.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 37 references that are in the map. First posted on arXiv 16 Jul 2026.

Builds on

34 of 37 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025DyNaVLM: Zero-Shot Vision-Language Navigation System with Dynamic Viewpoints and Self-Refining Graph Memory18 Jun
  2. 2025CL-CoTNav: Closed-Loop Hierarchical Chain-of-Thought for Zero-Shot Object-Goal Navigation with Vision-Language Models11 Apr
  3. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigation4 Mar
  4. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigation20 Feb
  5. 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering8 Nov
  6. 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Models16 Feb
  7. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation6 Dec

Other references · 27

  1. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  2. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  3. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  4. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  5. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  6. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
  7. 2018Semi-parametric Topological Memory for Navigationcited by 209
  8. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  9. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 178
  10. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  11. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 127
  12. 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 120
15 more
  1. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 117
  2. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 109
  3. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 94
  4. 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 77
  5. 2022Offline Visual Representation Learning for Embodied Navigationcited by 75
  6. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 70
  7. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 57
  8. 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 31
  9. 2025Open Scene Graphs for Open-World Object-Goal Navigationcited by 14
  10. 2024MemoNav: Working Memory Model for Visual Navigationcited by 13
  11. 2023ESceme: Vision-and-Language Navigation with Episodic Scene Memorycited by 10
  12. 2025C-NAV: Towards Self-Evolving Continual Object Navigation in Open Worldcited by 9
  13. 2026MerNav: A Highly Generalizable Memory-Execute-Review Framework for Zero-Shot Object Goal Navigationcited by 2
  14. 2026SSMG-Nav: Enhancing Lifelong Object Navigation with Semantic Skeleton Memory Graphcited by 2
  15. 2026GoalVLM: VLM-driven Object Goal Navigation for Multi-Agent Systemcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MemoryVLN: From Ego-Aware Memory Representation to Cross-Context Memory Reuse in Vision-Language Navigation
  2. 2026HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigation
  3. 2026EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigation
  4. 2026HiMemVLN: Enhancing Reliability of Open-Source Zero-Shot Vision-and-Language Navigation with Hierarchical Memory System
  5. 2026CGFM-Nav: Cognitive Graph-Field Memory for Semantic-Guided Lifelong Multimodal Embodied Navigation
  6. 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
  7. 2026CMMR-VLN: Vision-and-Language Navigation via Continual Multimodal Memory Retrieval
  8. 2026AgenticNav: Zero-Shot Vision-and-Language Navigation as a Tool-Calling Harness