mlsanityData to 14 Oct 2026

ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memory

arXiv 2607.10350 ↗First posted 11 Jul 2026cs.AI cs.ROcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Recent VLM and VLA systems have improved robotic perception and action prediction, yet long-horizon embodied agents still require a general runtime layer for reasoning, memory, tool use, verification, and cross-embodiment execution. We present ABot-AgentOS, a general robotic Agent Operating System that sits above low-level controllers and provides a deliberative agent layer for scene-conditioned planning, context-isolated skill execution, multi-stage verification, multi-modal memory, and edge-cloud collaboration. To evaluate such systems, we introduce EmbodiedWorldBench, an executable benchmark with 16 indoor, outdoor, and hybrid scenes, four difficulty levels, and over 200 tasks involving navigation, object search, NPC dialogue, dynamic events, and trace-grounded scoring. ABot-AgentOS further introduces Universal Multi-modal Graph Memory, a persistent source-grounded substrate that converts dialogue, visual observations, spatial context, temporal relations, and task traces into typed nodes and edges. A failure-driven self-evolution loop converts diagnosed memory failures into gated runtime evo-assets that are promoted only to later evaluation splits, preventing current-split ground-truth leakage while enabling continual improvement. On an initial EmbodiedWorldBench subset, ABot-AgentOS improves over a single-controller baseline in both task success and goal completion. Across memory benchmarks, ABot-AgentOS Static achieves 87.5 on LoCoMo, 59.9 on OpenEQA EM-EQA, 88.6 on Mem-Gallery, and 76.5 Acc@All on NExT-QA; self-evolution further improves LoCoMo to 88.7, OpenEQA to 60.4, and Mem-Gallery to 89.0. These results suggest that a general Agent OS layer can improve long-horizon embodied execution while providing persistent, auditable memory for continual interaction.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 99 of its 114 references that are in the map. First posted on arXiv 11 Jul 2026.

Builds on

99 of 114 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026When Robots Do the Chores: A Benchmark and Agent for Long-Horizon Household Task Execution14 May
  2. 2026Explore Like Humans: Autonomous Exploration with Online SG-Memo Construction for Embodied Agents21 Apr
  3. 2026Agentic Very Long Video Understanding26 Jan
  4. 2025WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning2 Dec
  5. 2025RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systems2 Aug
  6. 2025Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligence18 Jun
  7. 2025Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agents29 May
  8. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents13 Feb
  9. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environment12 Oct
  10. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding18 Mar

Other references · 89

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  8. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  9. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  10. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  11. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  12. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
38 more
  1. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  2. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  3. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  4. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  5. 2023Large Language Models as Optimizerscited by 864
  6. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  7. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  8. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  9. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 779
  10. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  11. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  12. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  13. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
  14. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 556
  15. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
  16. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
  17. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  18. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 228
  19. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 221
  20. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 220
  21. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 178
  22. 2026Qwen3.5-Omni Technical Reportcited by 174
  23. 2025MIRIX: Multi-Agent Memory System for LLM-Based Agentscited by 173
  24. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  25. 2022Revisiting the "Video" in Video-Language Understandingcited by 169
  26. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 165
  27. 2025Memory OS of AI Agentcited by 150
  28. 2026SimpleMem: Efficient Lifelong Memory for LLM Agentscited by 149
  29. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 142
  30. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 136
  31. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 135
  32. 2026ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learningcited by 103
  33. 2023AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learncited by 97
  34. 2025MemEvolve: Meta-Evolution of Agent Memory Systemscited by 95
  35. 2025Galaxea Open-World Dataset and G0 Dual-System VLA Modelcited by 92
  36. 2025EgoLife: Towards Egocentric Life Assistantcited by 91
  37. 2025RoboBrain 2.0 Technical Reportcited by 89
  38. 2025Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfercited by 87

50 most cited of 89 All 89 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ABot-Claw: A Foundation for Persistent, Cooperative, and Self-Evolving Robotic Agents
  2. 2026HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memory
  3. 2025RoboOS-NeXT: A Unified Memory-based Framework for Lifelong, Scalable, and Robust Multi-Robot Collaboration
  4. 2025FindingDory: A Benchmark to Evaluate Memory in Embodied Agents
  5. 2026PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution
  6. 2026What Should Embodied Agents Remember? Benchmarking and Evaluating Memory Modality for Embodied Agents
  7. 2026ChunkAct: Action Chunking for Agentic Robotics
  8. 2026RoboFoundry: System-as-Policy Evolution for Self-Learning Embodied Agents