mlsanityData to 7 Oct 2026

DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigation

arXiv 2505.21969 ↗First posted 28 May 2025cs.RO cs.AIcited by 6

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Adaptive navigation in unfamiliar environments is crucial for household service robots but remains challenging due to the need for both low-level path planning and high-level scene understanding. While recent vision-language model (VLM) based zero-shot approaches reduce dependence on prior maps and scene-specific training data, they face significant limitations: spatiotemporal discontinuity from discrete observations, unstructured memory representations, and insufficient task understanding leading to navigation failures. We propose DORAEMON (Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigation), a novel cognitive-inspired framework consisting of Ventral and Dorsal Streams that mimics human navigation capabilities. The Dorsal Stream implements the Hierarchical Semantic-Spatial Fusion and Topology Map to handle spatiotemporal discontinuities, while the Ventral Stream combines RAG-VLM and Policy-VLM to improve decision-making. Our approach also develops Nav-Ensurance to ensure navigation safety and efficiency. We evaluate DORAEMON on the HM3D, MP3D, and GOAT datasets, where it achieves state-of-the-art performance on both success rate (SR) and success weighted by path length (SPL) metrics, significantly outperforming existing methods. We also introduce a new evaluation metric (AORI) to assess navigation intelligence better. Comprehensive experiments demonstrate DORAEMON's effectiveness in zero-shot autonomous navigation without requiring prior map building or pre-training.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 46 of its 62 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

46 of 62 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022JARVIS: A Neuro-Symbolic Commonsense Reasoning Framework for Conversational Embodied Agents28 Aug

Other references · 45

  1. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  2. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  3. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  4. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  5. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  6. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
  7. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  8. 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 405
  9. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  10. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  11. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
  12. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
33 more
  1. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  2. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
  3. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
  4. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  5. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
  6. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 126
  7. 2021FILM: Following Instructions in Language with Modular Methodscited by 119
  8. 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 118
  9. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
  10. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  11. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 106
  12. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  13. 2023Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planningcited by 88
  14. 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 75
  15. 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 66
  16. 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 66
  17. 2024Does Spatial Cognition Emerge in Frontier Models?cited by 65
  18. 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 56
  19. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 55
  20. 2025MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigationcited by 55
  21. 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontierscited by 53
  22. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigationcited by 42
  23. 2024ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imaginationcited by 38
  24. 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answeringcited by 37
  25. 2024CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMscited by 32
  26. 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
  27. 2025ApexNav: An Adaptive Exploration Strategy for Zero-Shot Object Navigation with Target-centric Semantic Fusioncited by 29
  28. 2022Learning Active Camera for Multi-Object Navigationcited by 20
  29. 2024Zero-shot Object Navigation with Vision-Language Models Reasoningcited by 18
  30. 2024TopV-Nav: Unlocking the Top-View Spatial Reasoning Potential of MLLM for Zero-shot Object Navigationcited by 15
  31. 2025CL-CoTNav: Closed-Loop Hierarchical Chain-of-Thought for Zero-Shot Object-Goal Navigation with Vision-Language Modelscited by 11
  32. 2021Improving Safety in Deep Reinforcement Learning using Unsupervised Action Planningcited by 7
  33. 2025Endowing Embodied Agents with Spatial Reasoning Capabilities for Vision-and-Language Navigationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Deployable Embodied Vision-Language Navigation System with Hierarchical Cognition and Context-Aware Exploration ↗
  2. 2026A Semantic Autonomy Framework for VLM-Integrated Indoor Mobile Robots: Hybrid Deterministic Reasoning and Cross-Robot Adaptive Memory
  3. 2026HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigation
  4. 2026Explore Like Humans: Autonomous Exploration with Online SG-Memo Construction for Embodied Agents
  5. 2026CausalNav: A Long-term Embodied Navigation System for Autonomous Mobile Robots in Dynamic Outdoor Scenarios
  6. 2026SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning
  7. 2026Hierarchical Semantic-Augmented Navigation: Optimal Transport and Graph-Driven Reasoning for Vision-Language Navigation
  8. 2025DyNaVLM: Zero-Shot Vision-Language Navigation System with Dynamic Viewpoints and Self-Refining Graph Memory