# Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Making

arXiv: https://arxiv.org/abs/2505.19933 · HTML: https://mlsanity.com/paper/2505.19933/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 15 of 15)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://mlsanity.com/paper/2412.14470/) (2024)
- [R-Judge: Benchmarking Safety Risk Awareness for LLM Agents](https://mlsanity.com/paper/2401.10019/) (2024)
- [iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks](https://mlsanity.com/paper/2108.03272/) (2021)
- [BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments](https://mlsanity.com/paper/2108.03332/) (2021)
- [SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents](https://mlsanity.com/paper/2412.13178/) (2024)
- [Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents](https://mlsanity.com/paper/2309.09919/) (2023)
- [EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents](https://mlsanity.com/paper/2408.04449/) (2024)
- [Generating Symbolic World Models via Test-time Scaling of Large Language Models](https://mlsanity.com/paper/2502.04728/) (2025)
- [Safe Planner: Empowering Safety Awareness in Large Pre-Trained Models for Robot Task Planning](https://mlsanity.com/paper/2411.06920/) (2024)

## Built on by (most cited first, 20 of 20)

- [IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Tasks](https://mlsanity.com/paper/2506.16402/) (2025)
- [What Breaks Embodied AI Security:LLM Vulnerabilities, CPS Flaws,or Something Else?](https://mlsanity.com/paper/2602.17345/) (2026)
- [Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents](https://mlsanity.com/paper/2606.26080/) (2026)
- [HomeSafeBench: Benchmarking Embodied Vision-Language Models in Free-Exploration Home Safety Inspection](https://mlsanity.com/paper/2509.23690/) (2025)
- [HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task](https://mlsanity.com/paper/2603.14367/) (2026)
- [Using large language models for embodied planning introduces systematic safety risks](https://mlsanity.com/paper/2604.18463/) (2026)
- [Survival at Any Cost? LLMs and the Choice Between Self-Preservation and Human Harm](https://mlsanity.com/paper/2509.12190/) (2025)
- [EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents](https://mlsanity.com/paper/2605.30924/) (2026)
- [AURA: Affordance-Understanding and Risk-aware Alignment Technique for Large Language Models](https://mlsanity.com/paper/2508.06124/) (2025)
- [RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents](https://mlsanity.com/paper/2605.19328/) (2026)
- [SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents](https://mlsanity.com/paper/2607.14543/) (2026)
- [SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models](https://mlsanity.com/paper/2604.19638/) (2026)
- [Self-Evolving Just-In-Time Memory for Proactive Embodied Safety](https://mlsanity.com/paper/2607.16247/) (2026)
- [Safe Skill Retirement for Physical Agents](https://mlsanity.com/paper/2609.29543/) (2026)
- [GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI](https://mlsanity.com/paper/2608.21928/) (2026)
- [The Safeguard Worked. Is the LLM System Safer?](https://mlsanity.com/paper/2609.00519/) (2026)
- [STAGE: Diagnosing Semantic Transfer at Grounded Execution in Embodied Agents](https://mlsanity.com/paper/2609.13458/) (2026)
- [Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directions](https://mlsanity.com/paper/2609.27847/) (2026)
- [PlanGuard: A Guardrail for Multi-Step Plan Safety in Embodied Agents](https://mlsanity.com/paper/2609.32801/) (2026)
- [Refusals That Bend: Measuring and Predicting Task Malleability in Embodied VLM Planners](https://mlsanity.com/paper/2609.38971/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly used for decision making in embodied agents, yet existing safety evaluations often rely on coarse success rates and domain-specific setups, making it difficult to diagnose why and where these models fail. This obscures our understanding of embodied safety and limits the selective deployment of LLMs in high-risk physical environments. We introduce SAFEL, the framework for systematically evaluating the physical safety of LLMs in embodied decision making. SAFEL assesses two key competencies: (1) rejecting unsafe commands via the Command Refusal Test, and (2) generating safe and executable plans via the Plan Safety Test. Critically, the latter is decomposed into functional modules, goal interpretation, transition modeling, action sequencing, enabling fine-grained diagnosis of safety failures. To support this framework, we introduce EMBODYGUARD, a PDDL-grounded benchmark containing 942 LLM-generated scenarios covering both overtly malicious and contextually hazardous instructions. Evaluation across 13 state-of-the-art LLMs reveals that while models often reject clearly unsafe commands, they struggle to anticipate and mitigate subtle, situational risks. Our results highlight critical limitations in current LLMs and provide a foundation for more targeted, modular improvements in safe embodied reasoning.