RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
The dawn of embodied intelligence has ushered in an unprecedented imperative for resilient, cognition-enabled multi-agent collaboration across next-generation ecosystems, revolutionizing paradigms in autonomous manufacturing, adaptive service robotics, and cyber-physical production architectures. However, current robotic systems face significant limitations, such as limited cross-embodiment adaptability, inefficient task scheduling, and insufficient dynamic error correction. While End-to-end VLA models demonstrate inadequate long-horizon planning and task generalization, hierarchical VLA models suffer from a lack of cross-embodiment and multi-agent coordination capabilities. To address these challenges, we introduce RoboOS, the first open-source embodied system built on a Brain-Cerebellum hierarchical architecture, enabling a paradigm shift from single-agent to multi-agent intelligence. Specifically, RoboOS consists of three key components: (1) Embodied Brain Model (RoboBrain), a MLLM designed for global perception and high-level decision-making; (2) Cerebellum Skill Library, a modular, plug-and-play toolkit that facilitates seamless execution of multiple skills; and (3) Real-Time Shared Memory, a spatiotemporal synchronization mechanism for coordinating multi-agent states. By integrating hierarchical information flow, RoboOS bridges Embodied Brain and Cerebellum Skill Library, facilitating robust planning, scheduling, and error correction for long-horizon tasks, while ensuring efficient multi-agent collaboration through Real-Time Shared Memory. Furthermore, we enhance edge-cloud communication and cloud-based distributed inference to facilitate high-frequency interactions and enable scalable deployment. Extensive real-world experiments across various scenarios, demonstrate RoboOS's versatility in supporting heterogeneous embodiments. Project website: https://github.com/FlagOpen/RoboOS
Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 61 of its 84 references that are in the map. First posted on arXiv 6 May 2025.
Builds on
61 of 84 references.
Closest ancestors · 1
Same direction, newest first
- 2023An Embodied Generalist Agent in 3D World18 Nov
Other references · 60
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
38 more
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
- 2016Modeling Context in Referring Expressionscited by 1,054
- 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
- 2024PaliGemma: A versatile 3B VLM for transfercited by 679
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
- 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 439
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
- 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
- 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
- 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
- 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
- 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
- 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 242
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
- 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 216
- 2022AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domainscited by 205
- 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 195
- 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 187
- 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
- 2023RoCo: Dialectic Multi-Robot Collaboration with Large Language Modelscited by 171
- 2023PACO: Parts and Attributes of Common Objectscited by 146
- 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 134
- 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 131
- 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
- 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 116
- 2024LLARVA: Vision-Action Instruction Tuning Enhances Robot Learningcited by 87
- 2020Graph-Structured Referring Expression Reasoning in The Wildcited by 58
- 2025MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigationcited by 55
- 2024Revisiting Referring Expression Comprehension Evaluation in the Era of Large Multimodal Modelscited by 52
- 2020Cops-Ref: A new Dataset and Task on Compositional Referring Expression Comprehensioncited by 50
- 2025TLA: Tactile-Language-Action Model for Contact-Rich Manipulationcited by 46
- 2024COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Modelscited by 43
- 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 43
- 2024MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotationscited by 40
- 2025DexGrasp Anything: Towards Universal Robotic Dexterous Grasping with Physics Awarenesscited by 25
- 2022Grounded Affordance from Exocentric Viewcited by 21
50 most cited of 60 All 60 (JSON)
Built on by
Citing papers per year: 2025: 12; 2026: 16.
28 papers.
Sort:
By area:
- 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 125
- 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
- 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulationcited by 37
- 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
- 2025Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Braincited by 24
- 2025RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systemscited by 14
- 2025Towards Cross-View Point Correspondence in Vision-Language Modelscited by 12
- 2025Towards Spatial Trace with Reasoning in Vision-Language Models for Roboticscited by 12
- 2025VisualTrans: A Benchmark for Real-World Visual Transformation Reasoningcited by 9
- 2026What Matters in Orchestrating Robot Policies: A Systematic Study of Hierarchical VLA Agentscited by 9
- 2026RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Roboticscited by 6
- 2026AEROS: A Single-Agent Operating Architecture with Embodied Capability Modulescited by 6
16 more
- 2026Benchmark Everything Everywhere All at Oncecited by 5
- 2026PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Executioncited by 5
- 2025MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzlescited by 4
- 2025RoboNeuron: A Middle-Layer Infrastructure for Agent-Driven Orchestration in Embodied AIcited by 3
- 2025P3: Toward Versatile Embodied Agentscited by 2
- 2026BrainMem: Brain-Inspired Evolving Memory for Embodied Agent Task Planningcited by 2
- 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulationcited by 2
- 2026Advances and Innovations in the Multi-Agent Robotic System (MARS) Challengecited by 1
- 2026CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environmentcited by 1
- 2026HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memorycited by 1
- 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligencecited by 1
- 2026Federated Single-Agent Robotics: Multi-Robot Coordination Without Intra-Robot Multi-Agent Fragmentationcited by 0
- 2026Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futurescited by 0
- 2026MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Interventioncited by 0
- 2026IMPACT: Learning Internal-Model Predictive Control for Forceful Robotic Manipulationcited by 0
- 2026ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordinationcited by 0
- 2026ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordination21 Sep 2026
- 2026PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution18 Jul 2026
- 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation15 Jul 2026
- 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence13 Jul 2026
- 2026HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memory22 Jun 2026
- 2026IMPACT: Learning Internal-Model Predictive Control for Forceful Robotic Manipulation9 Jun 2026
- 2026What Matters in Orchestrating Robot Policies: A Systematic Study of Hierarchical VLA Agents9 Jun 2026
- 2026Benchmark Everything Everywhere All at Once4 Jun 2026
- 2026MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention31 May 2026
- 2026Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futures20 Apr 2026
- 2026Federated Single-Agent Robotics: Multi-Robot Coordination Without Intra-Robot Multi-Agent Fragmentation13 Apr 2026
- 2026AEROS: A Single-Agent Operating Architecture with Embodied Capability Modules8 Apr 2026
16 more
- 2026CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environment7 Apr 2026
- 2026RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics13 Mar 2026
- 2026BrainMem: Brain-Inspired Evolving Memory for Embodied Agent Task Planning12 Mar 2026
- 2026Advances and Innovations in the Multi-Agent Robotic System (MARS) Challenge26 Jan 2026
- 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulation29 Dec 2025
- 2025Towards Spatial Trace with Reasoning in Vision-Language Models for Robotics15 Dec 2025
- 2025RoboNeuron: A Middle-Layer Infrastructure for Agent-Driven Orchestration in Embodied AI11 Dec 2025
- 2025Towards Cross-View Point Correspondence in Vision-Language Models4 Dec 2025
- 2025Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain20 Oct 2025
- 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Robotics8 Oct 2025
- 2025MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzles1 Oct 2025
- 2025P3: Toward Versatile Embodied Agents9 Aug 2025
- 2025VisualTrans: A Benchmark for Real-World Visual Transformation Reasoning6 Aug 2025
- 2025RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systems2 Aug 2025
- 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspective2 Jul 2025
- 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics4 Jun 2025
- 2025RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systemscited by 14
- 2025P3: Toward Versatile Embodied Agentscited by 2
- 2026BrainMem: Brain-Inspired Evolving Memory for Embodied Agent Task Planningcited by 2
- 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulationcited by 2
- 2026CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environmentcited by 1
- 2026HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memorycited by 1
- 2026Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futurescited by 0
- 2026ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordinationcited by 0
- 2026ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordination21 Sep 2026
- 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation15 Jul 2026
- 2026HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memory22 Jun 2026
- 2026Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futures20 Apr 2026
- 2026CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environment7 Apr 2026
- 2026BrainMem: Brain-Inspired Evolving Memory for Embodied Agent Task Planning12 Mar 2026
- 2025P3: Toward Versatile Embodied Agents9 Aug 2025
- 2025RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systems2 Aug 2025
- 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 125
- 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulationcited by 37
- 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
- 2025Towards Spatial Trace with Reasoning in Vision-Language Models for Roboticscited by 12
- 2025VisualTrans: A Benchmark for Real-World Visual Transformation Reasoningcited by 9
- 2026RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Roboticscited by 6
- 2025MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzlescited by 4
- 2026RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics13 Mar 2026
- 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulation29 Dec 2025
- 2025Towards Spatial Trace with Reasoning in Vision-Language Models for Robotics15 Dec 2025
- 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Robotics8 Oct 2025
- 2025MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzles1 Oct 2025
- 2025VisualTrans: A Benchmark for Real-World Visual Transformation Reasoning6 Aug 2025
- 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics4 Jun 2025
- 2026AEROS: A Single-Agent Operating Architecture with Embodied Capability Modulescited by 6
- 2026Benchmark Everything Everywhere All at Oncecited by 5
- 2026Federated Single-Agent Robotics: Multi-Robot Coordination Without Intra-Robot Multi-Agent Fragmentationcited by 0
- 2026MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Interventioncited by 0
- 2026IMPACT: Learning Internal-Model Predictive Control for Forceful Robotic Manipulationcited by 0
- 2026IMPACT: Learning Internal-Model Predictive Control for Forceful Robotic Manipulation9 Jun 2026
- 2026Benchmark Everything Everywhere All at Once4 Jun 2026
- 2026MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention31 May 2026
- 2026Federated Single-Agent Robotics: Multi-Robot Coordination Without Intra-Robot Multi-Agent Fragmentation13 Apr 2026
- 2026AEROS: A Single-Agent Operating Architecture with Embodied Capability Modules8 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2025RoboOS-NeXT: A Unified Memory-based Framework for Lifelong, Scalable, and Robust Multi-Robot Collaboration
- 2026When Multi-Robot Systems Meet Agentic AI:Towards Embodied Collective Intelligence
- 2025Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review
- 2024BBSEA: An Exploration of Brain-Body Synchronization for Embodied Agents
- 2025Transforming Monolithic Foundation Models into Embodied Multi-Agent Architectures for Human-Robot Collaboration
- 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI
- 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Models
- 2026ABot-Claw: A Foundation for Persistent, Cooperative, and Self-Evolving Robotic Agents