mlsanityData to 14 Oct 2026

EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems

arXiv 2604.11174 ↗First posted 13 Apr 2026cs.RO cs.AIcited by 2

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Recent progress in embodied AI has produced a growing ecosystem of robot policies, foundation models, and modular runtimes. However, current evaluation remains dominated by task success metrics such as completion rate or manipulation accuracy. These metrics leave a critical gap: they do not measure whether embodied systems are governable -- whether they respect capability boundaries, enforce policies, recover safely, maintain audit trails, and respond to human oversight. We present EmbodiedGovBench, a benchmark for governance-oriented evaluation of embodied agent systems. Rather than asking only whether a robot can complete a task, EmbodiedGovBench evaluates whether the system remains controllable, policy-bounded, recoverable, auditable, and evolution-safe under realistic perturbations. The benchmark covers seven governance dimensions: unauthorized capability invocation, runtime drift robustness, recovery success, policy portability, version upgrade safety, human override responsiveness, and audit completeness. We define a benchmark structure spanning single-robot and fleet settings, with scenario templates, perturbation operators, governance metrics, and baseline evaluation protocols. We describe how the benchmark can be instantiated over embodied capability runtimes with modular interfaces and contract-aware upgrade workflows. Our analysis suggests that embodied governance should become a first-class evaluation target. EmbodiedGovBench provides the initial measurement framework for that shift.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 56 of its 95 references that are in the map. First posted on arXiv 13 Apr 2026.

Builds on

56 of 95 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Tasks19 Jun
  2. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions17 Jun
  3. 2025A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agents20 Apr
  4. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agents19 Dec
  5. 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents17 Dec
  6. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
  7. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agents18 Jan
  8. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 48

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  5. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  9. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  10. 2016Concrete Problems in AI Safetycited by 1,642
  11. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  12. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
36 more
  1. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  2. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  3. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  4. 2017Constrained Policy Optimizationcited by 983
  5. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  6. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  7. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
  8. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  9. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  10. 2017Safe Reinforcement Learning via Shieldingcited by 449
  11. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  12. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 384
  13. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
  14. 2019Control Barrier Functions: Theory and Applicationscited by 372
  15. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  16. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  17. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  18. 2020Integrated Task and Motion Planningcited by 290
  19. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
  20. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 221
  21. 2021TEACh: Task-driven Embodied Agents that Chatcited by 197
  22. 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 148
  23. 2024Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Makingcited by 144
  24. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  25. 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemscited by 96
  26. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 94
  27. 2024PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Taskscited by 82
  28. 2023State-wise Safe Reinforcement Learning: A Surveycited by 55
  29. 2021Run Time Assurance for Safety-Critical Systems: An Introduction to Safety Filtering Approaches for Complex Control Systemscited by 27
  30. 2018SOTER: A Runtime Assurance Framework for Programming Safe Robotics Systemscited by 25
  31. 2025Safety of Embodied Navigation: A Surveycited by 10
  32. 2026Harnessing Embodied Agents: Runtime Governance for Policy-Constrained Executioncited by 7
  33. 2025Embodied AI: Emerging Risks and Opportunities for Policy Actioncited by 5
  34. 2021Auditing Robot Learning for Safety and Compliance during Deploymentcited by 4
  35. 2026Evolving Skill Modules under a Fixed Planner: Versioning, Rollback, and Runtime Governance for Long-Lived Robot Systemscited by 4
  36. 2026ECM Contracts: Contract-Aware, Versioned, and Governable Capability Interfaces for Embodied Agentscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Digital to Physical: Digital Agents as Autonomous Coaches for Physical Intelligence
  2. 2026Beyond Task Success: An Evidence-Synthesis Framework for Evaluating, Governing, and Orchestrating Agentic AI
  3. 2026RoboFoundry: System-as-Policy Evolution for Self-Learning Embodied Agents
  4. 2026Embodied Skill Bench: Benchmarking How Well Agents with Embodied Skills Perform on Physical Tasks
  5. 2026Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses
  6. 2026Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement
  7. 2026RT-Safe: Benchmarking Agent Safety in Real-Time Embodied Environment
  8. 2025Generative AI in Embodied Systems: System-Level Analysis of Performance, Efficiency and Scalability