DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigation
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Adaptive navigation in unfamiliar environments is crucial for household service robots but remains challenging due to the need for both low-level path planning and high-level scene understanding. While recent vision-language model (VLM) based zero-shot approaches reduce dependence on prior maps and scene-specific training data, they face significant limitations: spatiotemporal discontinuity from discrete observations, unstructured memory representations, and insufficient task understanding leading to navigation failures. We propose DORAEMON (Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigation), a novel cognitive-inspired framework consisting of Ventral and Dorsal Streams that mimics human navigation capabilities. The Dorsal Stream implements the Hierarchical Semantic-Spatial Fusion and Topology Map to handle spatiotemporal discontinuities, while the Ventral Stream combines RAG-VLM and Policy-VLM to improve decision-making. Our approach also develops Nav-Ensurance to ensure navigation safety and efficiency. We evaluate DORAEMON on the HM3D, MP3D, and GOAT datasets, where it achieves state-of-the-art performance on both success rate (SR) and success weighted by path length (SPL) metrics, significantly outperforming existing methods. We also introduce a new evaluation metric (AORI) to assess navigation intelligence better. Comprehensive experiments demonstrate DORAEMON's effectiveness in zero-shot autonomous navigation without requiring prior map building or pre-training.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 46 of its 62 references that are in the map. First posted on arXiv 28 May 2025.
Builds on
46 of 62 references.
Closest ancestors · 1
Same direction, newest first
- 2022JARVIS: A Neuro-Symbolic Commonsense Reasoning Framework for Conversational Embodied Agents28 Aug
Other references · 45
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
- 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
- 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
- 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 405
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
- 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
- 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
- 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
33 more
- 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
- 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
- 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
- 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
- 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
- 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 126
- 2021FILM: Following Instructions in Language with Modular Methodscited by 119
- 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 118
- 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
- 2022Habitat-Matterport 3D Semantics Datasetcited by 109
- 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 106
- 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
- 2023Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planningcited by 88
- 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 75
- 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 66
- 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 66
- 2024Does Spatial Cognition Emerge in Frontier Models?cited by 65
- 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 56
- 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 55
- 2025MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigationcited by 55
- 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontierscited by 53
- 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigationcited by 42
- 2024ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imaginationcited by 38
- 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answeringcited by 37
- 2024CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMscited by 32
- 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
- 2025ApexNav: An Adaptive Exploration Strategy for Zero-Shot Object Navigation with Target-centric Semantic Fusioncited by 29
- 2022Learning Active Camera for Multi-Object Navigationcited by 20
- 2024Zero-shot Object Navigation with Vision-Language Models Reasoningcited by 18
- 2024TopV-Nav: Unlocking the Top-View Spatial Reasoning Potential of MLLM for Zero-shot Object Navigationcited by 15
- 2025CL-CoTNav: Closed-Loop Hierarchical Chain-of-Thought for Zero-Shot Object-Goal Navigation with Vision-Language Modelscited by 11
- 2021Improving Safety in Deep Reinforcement Learning using Unsupervised Action Planningcited by 7
- 2025Endowing Embodied Agents with Spatial Reasoning Capabilities for Vision-and-Language Navigationcited by 4
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Explorationcited by 13
- 2026TPRU: Advancing Temporal and Procedural Understanding in Large Multimodal Modelscited by 2
- 2026ReMemNav: Memory-Based Decision Correction and Target Verification for Zero-Shot Object Navigationcited by 2
- 2026HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memorycited by 1
- 2026World2Minecraft: Occupancy-Driven Simulated Scenes Constructioncited by 0
- 2026AnyGoal: Vision-Language Guided Multi-Agent Exploration for Training-Free Lifelong Navigationcited by 0
- 2026HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memory22 Jun 2026
- 2026AnyGoal: Vision-Language Guided Multi-Agent Exploration for Training-Free Lifelong Navigation11 Jun 2026
- 2026World2Minecraft: Occupancy-Driven Simulated Scenes Construction30 Apr 2026
- 2026ReMemNav: Memory-Based Decision Correction and Target Verification for Zero-Shot Object Navigation25 Mar 2026
- 2026TPRU: Advancing Temporal and Procedural Understanding in Large Multimodal Models21 Feb 2026
- 2026Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Exploration11 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2026A Deployable Embodied Vision-Language Navigation System with Hierarchical Cognition and Context-Aware Exploration ↗
- 2026A Semantic Autonomy Framework for VLM-Integrated Indoor Mobile Robots: Hybrid Deterministic Reasoning and Cross-Robot Adaptive Memory
- 2026HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigation
- 2026Explore Like Humans: Autonomous Exploration with Online SG-Memo Construction for Embodied Agents
- 2026CausalNav: A Long-term Embodied Navigation System for Autonomous Mobile Robots in Dynamic Outdoor Scenarios
- 2026SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning
- 2026Hierarchical Semantic-Augmented Navigation: Optimal Transport and Graph-Driven Reasoning for Vision-Language Navigation
- 2025DyNaVLM: Zero-Shot Vision-Language Navigation System with Dynamic Viewpoints and Self-Refining Graph Memory