mlsanityData to 14 Oct 2026

HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memory

arXiv 2606.23565 ↗First posted 22 Jun 2026cs.RO cs.CVcited by 1

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

LLM agents follow a practical execution loop in digital environments: they reason over structured states, invoke tools, inspect feedback, and revise actions. Extending this loop to physical robots is difficult because physical execution is continuous, embodiment-dependent, uncertain, and constrained by safety. Existing embodied-AI systems have advanced manipulation, spatial understanding, navigation, and humanoid control, but these capabilities often remain specialized modules or loosely coupled decision loops. In this work, we introduce HoloAgent-0, a unified embodied agent framework for real-world robot deployment. Embodied AgentOS converts language instructions into executable skill graphs, schedules robot resources, monitors execution, and triggers clarification or re-planning from runtime feedback. HoloAgent-0 organizes heterogeneous robot models and controllers through three coupled layers: Embodied AgentOS for closed-loop execution, 3D spatial memory for physical world grounding, and embodied skills for robot action. We deploy HoloAgent-0 on real hardware and evaluate its spatial memory, long-horizon navigation, and closed-loop execution across motion generation, object search, cross-robot coordination, and mobile manipulation.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 64 of its 68 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

64 of 68 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigation28 May
  2. 2025RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration6 May
  3. 2024EMOS: Embodiment-aware Heterogeneous Multi-robot Operating System with LLM Agents30 Oct

Other references · 61

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  9. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  10. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  11. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  12. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
38 more
  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  2. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  3. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  5. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
  6. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  7. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 519
  8. 2026World Action Models are Zero-shot Policiescited by 465
  9. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  10. 2022Visual Language Maps for Robot Navigationcited by 365
  11. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
  12. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
  13. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  14. 2026Causal World Modeling for Robot Controlcited by 321
  15. 2020Integrated Task and Motion Planningcited by 290
  16. 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 281
  17. 2025Motus: A Unified Latent Action World Modelcited by 251
  18. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
  19. 2022Robot Operating System 2: Design, Architecture, and Uses In The Wildcited by 199
  20. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 196
  21. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 166
  22. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
  23. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 138
  24. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 109
  25. 2025SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Controlcited by 101
  26. 2024What Matters in Building Vision-Language-Action Models for Generalist Robotscited by 89
  27. 2025GMT: General Motion Tracking for Humanoid Whole-Body Controlcited by 83
  28. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  29. 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphscited by 73
  30. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigationcited by 43
  31. 2025EmbodiedGen: Towards a Generative 3D World Engine for Embodied Intelligencecited by 36
  32. 2025Track Any Motions under Any Disturbancescited by 35
  33. 2024ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoningcited by 27
  34. 2022FAST-LIVO: Fast and Tightly-coupled Sparse-Direct LiDAR-Inertial-Visual Odometrycited by 24
  35. 2025Aux-Think: Exploring Reasoning Strategies for Data-Efficient Vision-Language Navigationcited by 21
  36. 2026HoloBrain-0 Technical Reportcited by 21
  37. 2025MonoDream: Monocular Vision-Language Navigation with Panoramic Dreamingcited by 16
  38. 2025MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigationcited by 13

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI
  2. 2026ABot-Claw: A Foundation for Persistent, Cooperative, and Self-Evolving Robotic Agents
  3. 2026ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memory
  4. 2026ETA: A New Agentic Paradigm for Embodied Tasks
  5. 2026When Multi-Robot Systems Meet Agentic AI:Towards Embodied Collective Intelligence
  6. 2026Embodied Agents Take Control: Minimal-Interface Zero-Shot Agents Rival Industrial-Scale Policies in Vision-and-Language Navigation
  7. 2026RoboBridge: A Self-Evolving Embodied Agent Framework for Sim-to-Real Transfer
  8. 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation