mlsanityData to 7 Oct 2026

RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration

arXiv 2505.03673 ↗First posted 6 May 2025cs.ROcited by 28

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

The dawn of embodied intelligence has ushered in an unprecedented imperative for resilient, cognition-enabled multi-agent collaboration across next-generation ecosystems, revolutionizing paradigms in autonomous manufacturing, adaptive service robotics, and cyber-physical production architectures. However, current robotic systems face significant limitations, such as limited cross-embodiment adaptability, inefficient task scheduling, and insufficient dynamic error correction. While End-to-end VLA models demonstrate inadequate long-horizon planning and task generalization, hierarchical VLA models suffer from a lack of cross-embodiment and multi-agent coordination capabilities. To address these challenges, we introduce RoboOS, the first open-source embodied system built on a Brain-Cerebellum hierarchical architecture, enabling a paradigm shift from single-agent to multi-agent intelligence. Specifically, RoboOS consists of three key components: (1) Embodied Brain Model (RoboBrain), a MLLM designed for global perception and high-level decision-making; (2) Cerebellum Skill Library, a modular, plug-and-play toolkit that facilitates seamless execution of multiple skills; and (3) Real-Time Shared Memory, a spatiotemporal synchronization mechanism for coordinating multi-agent states. By integrating hierarchical information flow, RoboOS bridges Embodied Brain and Cerebellum Skill Library, facilitating robust planning, scheduling, and error correction for long-horizon tasks, while ensuring efficient multi-agent collaboration through Real-Time Shared Memory. Furthermore, we enhance edge-cloud communication and cloud-based distributed inference to facilitate high-frequency interactions and enable scalable deployment. Extensive real-world experiments across various scenarios, demonstrate RoboOS's versatility in supporting heterogeneous embodiments. Project website: https://github.com/FlagOpen/RoboOS

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 61 of its 84 references that are in the map. First posted on arXiv 6 May 2025.

Builds on

61 of 84 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023An Embodied Generalist Agent in 3D World18 Nov

Other references · 60

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  6. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  9. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  10. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  11. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  12. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
38 more
  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  2. 2016Modeling Context in Referring Expressionscited by 1,054
  3. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  4. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  5. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  6. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  7. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 439
  8. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  9. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
  10. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  11. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
  12. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  13. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  14. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
  15. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 242
  16. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  17. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 216
  18. 2022AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domainscited by 205
  19. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 195
  20. 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 187
  21. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  22. 2023RoCo: Dialectic Multi-Robot Collaboration with Large Language Modelscited by 171
  23. 2023PACO: Parts and Attributes of Common Objectscited by 146
  24. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 134
  25. 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 131
  26. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  27. 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 116
  28. 2024LLARVA: Vision-Action Instruction Tuning Enhances Robot Learningcited by 87
  29. 2020Graph-Structured Referring Expression Reasoning in The Wildcited by 58
  30. 2025MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigationcited by 55
  31. 2024Revisiting Referring Expression Comprehension Evaluation in the Era of Large Multimodal Modelscited by 52
  32. 2020Cops-Ref: A new Dataset and Task on Compositional Referring Expression Comprehensioncited by 50
  33. 2025TLA: Tactile-Language-Action Model for Contact-Rich Manipulationcited by 46
  34. 2024COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Modelscited by 43
  35. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 43
  36. 2024MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotationscited by 40
  37. 2025DexGrasp Anything: Towards Universal Robotic Dexterous Grasping with Physics Awarenesscited by 25
  38. 2022Grounded Affordance from Exocentric Viewcited by 21

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 16.

28 papers.

Sort:

By area:

  1. 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 125
  2. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  3. 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulationcited by 37
  4. 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
  5. 2025Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Braincited by 24
  6. 2025RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systemscited by 14
  7. 2025Towards Cross-View Point Correspondence in Vision-Language Modelscited by 12
  8. 2025Towards Spatial Trace with Reasoning in Vision-Language Models for Roboticscited by 12
  9. 2025VisualTrans: A Benchmark for Real-World Visual Transformation Reasoningcited by 9
  10. 2026What Matters in Orchestrating Robot Policies: A Systematic Study of Hierarchical VLA Agentscited by 9
  11. 2026RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Roboticscited by 6
  12. 2026AEROS: A Single-Agent Operating Architecture with Embodied Capability Modulescited by 6
16 more
  1. 2026Benchmark Everything Everywhere All at Oncecited by 5
  2. 2026PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Executioncited by 5
  3. 2025MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzlescited by 4
  4. 2025RoboNeuron: A Middle-Layer Infrastructure for Agent-Driven Orchestration in Embodied AIcited by 3
  5. 2025P3: Toward Versatile Embodied Agentscited by 2
  6. 2026BrainMem: Brain-Inspired Evolving Memory for Embodied Agent Task Planningcited by 2
  7. 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulationcited by 2
  8. 2026Advances and Innovations in the Multi-Agent Robotic System (MARS) Challengecited by 1
  9. 2026CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environmentcited by 1
  10. 2026HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memorycited by 1
  11. 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligencecited by 1
  12. 2026Federated Single-Agent Robotics: Multi-Robot Coordination Without Intra-Robot Multi-Agent Fragmentationcited by 0
  13. 2026Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futurescited by 0
  14. 2026MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Interventioncited by 0
  15. 2026IMPACT: Learning Internal-Model Predictive Control for Forceful Robotic Manipulationcited by 0
  16. 2026ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordinationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RoboOS-NeXT: A Unified Memory-based Framework for Lifelong, Scalable, and Robust Multi-Robot Collaboration
  2. 2026When Multi-Robot Systems Meet Agentic AI:Towards Embodied Collective Intelligence
  3. 2025Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review
  4. 2024BBSEA: An Exploration of Brain-Body Synchronization for Embodied Agents
  5. 2025Transforming Monolithic Foundation Models into Embodied Multi-Agent Architectures for Human-Robot Collaboration
  6. 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI
  7. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Models
  8. 2026ABot-Claw: A Foundation for Persistent, Cooperative, and Self-Evolving Robotic Agents