PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution
On the map: World models → World models for language agents
also in 1 more
Abstract (arXiv)
Vision-language-action models, world models, and agentic planners each advance physical intelligence, yet their composition lacks a common execution abstraction, shared state, semantic verification, and persistent experience across heterogeneous embodiments. We present PhyAgentOS, a runtime foundation delivering scheduling, verification, memory, benchmarking, and safety as system-level services. Its Session-Centered Runtime treats a session, not an action, as the minimum unit of scheduling, compatibility preflight, supervised execution, evidence collection, and acceptance. To decouple cognition from physical execution, the cognition-physics boundary is a file system: the State-as-a-File protocol materializes cross-layer state as Markdown with YAML, yielding inspectable, versionable records without code dependencies between Agent and Runtime layers. These views form a unified cognitive state space aligning intent, capabilities, environment, execution, and experience. The SessionVerifier distinguishes execution termination from semantic task completion via evidence-grounded verdicts of success, failure, or replan. Verified outcomes are consolidated through epistemic memory into reusable knowledge and corrective lessons, closing a trial-and-error loop without retraining. Benchmarking reuses the deployment session and verification path, so results trace to real execution. Layered safety constrains both policy-driven and agent-driven execution: preflight, action bridges, SafetyGuard, heartbeat monitoring, and target-local constraints. Validation is progressive: games test cognitive planning, simulation adds dynamics and control, real robots add hardware noise, with the cognitive layer held constant. PhyAgentOS is benchmarked on Optimus-67, StarDojo, and DST-Dojo, validated on 19+ simulated and physical embodiments, and gains on LIBERO, Calvin, and RoboCasa365 across multiple VLA models.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 53 of its 136 references that are in the map. First posted on arXiv 18 Jul 2026.
Builds on
53 of 136 references.
References · 53
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
38 more
- 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
- 2022A Generalist Agentcited by 802
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
- 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
- 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
- 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
- 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
- 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
- 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
- 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
- 2022DayDreamer: World Models for Physical Robot Learningcited by 372
- 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
- 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
- 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
- 2024RT-H: Action Hierarchies Using Languagecited by 204
- 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
- 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
- 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 180
- 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 171
- 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
- 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 155
- 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 128
- 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 120
- 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 116
- 2024AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agentscited by 83
- 2025Gemini Robotics: Bringing AI into the Physical Worldcited by 77
- 2026CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulationcited by 72
- 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulatorscited by 65
- 2024Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detectioncited by 59
- 2026ASPIRE: Agentic /Skills Discovery for Roboticscited by 44
- 2020Model-based Reinforcement Learning: A Surveycited by 39
- 2025RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaborationcited by 28
- 2026RLDX-1 Technical Reportcited by 25
- 2026World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetrycited by 13
- 2025Optimus-3: Dual-Router Aligned Mixture-of-Experts Agent with Dual-Granularity Reasoning-Aware Policy Optimizationcited by 12
- 2025StarDojo: Benchmarking Open-Ended Behaviors of Agentic Multimodal LLMs in Production-Living Simulations with Stardew Valleycited by 7
- 2026ABot-Claw: A Foundation for Persistent, Cooperative, and Self-Evolving Robotic Agentscited by 6
- 2025Learning to See and Act: Task-Aware Virtual View Exploration for Robotic Manipulationcited by 5
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026HarnessPAI: An Evolving Harness for Physical AIcited by 1
- 2026Teach and Grow: An Agent-Centered Architecture for General Robot Learningcited by 0
- 2026ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordinationcited by 0
- 2026World Agent: Can Language Models Keep a World Running?cited by 0
- 2026DynaHarness: A Dynamic Physical Harness for Self-Evolving Robot Agentscited by 0
- 2026DynaHarness: A Dynamic Physical Harness for Self-Evolving Robot Agents30 Sep 2026
- 2026World Agent: Can Language Models Keep a World Running?26 Sep 2026
- 2026HarnessPAI: An Evolving Harness for Physical AI24 Sep 2026
- 2026ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordination21 Sep 2026
- 2026Teach and Grow: An Agent-Centered Architecture for General Robot Learning17 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026TypeGo: An OS Runtime for Embodied Agents
- 2026RPMS: Enhancing LLM-Based Embodied Planning through Rule-Augmented Memory Synergy
- 2026RoboFoundry: System-as-Policy Evolution for Self-Learning Embodied Agents
- 2026Towards a Systems Foundation for Agentic Skills: Architecture, Lifecycle, and Security
- 2026ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memory
- 2026Architecting AgentOS: From Token-Level Context to Emergent System-Level Intelligence
- 2026Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy
- 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence