# EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents arXiv: https://arxiv.org/abs/2408.04449 ยท HTML: https://mlsanity.com/paper/2408.04449/ ## Key facts - Cited by 27 papers in the map (data through 14 Oct 2026). - Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. - Builds on 8 of its 35 references that are in the map. - First posted on arXiv 8 Aug 2024. ## Placed in - LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/) ## Builds on (most cited first, 8 of 8) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023) - [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024) - [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022) - [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024) - [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024) - [Open-Ended Instructable Embodied Agents with Memory-Augmented Large Language Models](https://mlsanity.com/paper/2310.15127/) (2023) - [Leveraging Large Language Models for enhanced personalised user experience in Smart Homes](https://mlsanity.com/paper/2407.12024/) (2024) ## Built on by (most cited first, 27 of 27) - [SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents](https://mlsanity.com/paper/2412.13178/) (2024) - [IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Tasks](https://mlsanity.com/paper/2506.16402/) (2025) - [AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions](https://mlsanity.com/paper/2506.14697/) (2025) - [Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Making](https://mlsanity.com/paper/2505.19933/) (2025) - [SafeMind: Benchmarking and Mitigating Safety Risks in Embodied LLM Agents](https://mlsanity.com/paper/2509.25885/) (2025) - [Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges](https://mlsanity.com/paper/2507.19672/) (2025) - [Safety of Embodied Navigation: A Survey](https://mlsanity.com/paper/2508.05855/) (2025) - [Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation](https://mlsanity.com/paper/2606.10749/) (2026) - [HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task](https://mlsanity.com/paper/2603.14367/) (2026) - [ForesightSafety Bench: A Frontier Risk Evaluation and Governance Framework towards Safe AI](https://mlsanity.com/paper/2602.14135/) (2026) - [RoboInspector: Unveiling the Unreliability of Policy Code for LLM-enabled Robotic Manipulation](https://mlsanity.com/paper/2508.21378/) (2025) - [SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agents](https://mlsanity.com/paper/2510.12985/) (2025) - [RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents](https://mlsanity.com/paper/2605.19328/) (2026) - [Benchmarking the Safety of Large Language Models for Robotic Health Attendant Control](https://mlsanity.com/paper/2604.26577/) (2026) - [Safe Embodied AI for Long-horizon Tasks: A Cross-layer Analysis of Robotic Manipulation](https://mlsanity.com/paper/2606.05660/) (2026) - [Self-Evolving Just-In-Time Memory for Proactive Embodied Safety](https://mlsanity.com/paper/2607.16247/) (2026) - [Towards provable probabilistic safety for scalable embodied AI systems](https://mlsanity.com/paper/2506.05171/) (2025) - [Context-Aware Risk Estimation in Home Environments: A Probabilistic Framework for Service Robots](https://mlsanity.com/paper/2508.19788/) (2025) - [Beyond Model Jailbreak: Systematic Dissection of the "Ten DeadlySins" in Embodied Intelligence](https://mlsanity.com/paper/2512.06387/) (2025) - [ICAT: Incident-Case-Grounded Adaptive Testing for Physical-Risk Prediction in Embodied World Models](https://mlsanity.com/paper/2604.16405/) (2026) - [When Words Are Safe But Actions Kill: Probing Physical Jailbreak Beyond Textual Jailbreak in Hidden-State Risk Space](https://mlsanity.com/paper/2607.15218/) (2026) - [GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI](https://mlsanity.com/paper/2608.21928/) (2026) - [Resilience Matters for Embodied Agents System: New Metrics, Systematic Evaluation, and Optimization](https://mlsanity.com/paper/2608.23839/) (2026) - [CLAMP: Constrained Decoding for Vision-Language Embodied Planning](https://mlsanity.com/paper/2609.08602/) (2026) - [Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directions](https://mlsanity.com/paper/2609.27847/) (2026) - [PlanGuard: A Guardrail for Multi-Step Plan Safety in Embodied Agents](https://mlsanity.com/paper/2609.32801/) (2026) - [Refusals That Bend: Measuring and Predicting Task Malleability in Embodied VLM Planners](https://mlsanity.com/paper/2609.38971/) (2026) ## Abstract (arXiv) Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction. The emergence of foundation models as the "brain" of EAI agents for high-level task planning has shown promising results. However, the deployment of these agents in physical environments presents significant safety challenges. For instance, a housekeeping robot lacking sufficient risk awareness might place a metal container in a microwave, potentially causing a fire. To address these critical safety concerns, comprehensive pre-deployment risk assessments are imperative. This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios. EAIRiskBench employs a multi-agent cooperative system that leverages various foundation models to generate safety guidelines, create risk-prone scenarios, make task planning, and evaluate safety systematically. Utilizing this framework, we construct EARDataset, comprising diverse test cases across various domains, encompassing both textual and visual scenarios. Our comprehensive evaluation of state-of-the-art foundation models reveals alarming results: all models exhibit high task risk rates (TRR), with an average of 95.75% across all evaluated models. To address these challenges, we further propose two prompting-based risk mitigation strategies. While these strategies demonstrate some efficacy in reducing TRR, the improvements are limited, still indicating substantial safety concerns. This study provides the first large-scale assessment of physical risk awareness in EAI agents. Our findings underscore the critical need for enhanced safety measures in EAI systems and provide valuable insights for future research directions in developing safer embodied artificial intelligence system. Data and code are available at https://github.com/zihao-ai/EARBench.