Can AI Perceive Physical Danger and Intervene?
Not on the map. Why?
Abstract (arXiv)
When AI interacts with the physical world -- as a robot or an assistive agent -- new safety challenges emerge beyond those of purely ``digital AI". In such interactions, the potential for physical harm is direct and immediate. How well do state-of-the-art foundation models understand common-sense facts about physical safety, e.g. that a box may be too heavy to lift, or that a hot cup of coffee should not be handed to a child? In this paper, our contributions are three-fold: first, we develop a highly scalable approach to continuous physical safety benchmarking of Embodied AI systems, grounded in real-world injury narratives and operational safety constraints. To probe multi-modal safety understanding, we turn these narratives and constraints into photorealistic images and videos capturing transitions from safe to unsafe states, using advanced generative models. Secondly, we comprehensively analyze the ability of major foundation models to perceive risks, reason about safety, and trigger interventions; this yields multi-faceted insights into their deployment readiness for safety-critical agentic applications. Finally, we develop a post-training paradigm to teach models to explicitly reason about embodiment-specific safety constraints provided through system instructions. The resulting models generate thinking traces that make safety reasoning interpretable and transparent, achieving state of the art performance in constraint satisfaction evaluations. The benchmark is released at https://asimov-benchmark.github.io/v2
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 25 Sep 2025.
Builds on
26 of 42 references.
References · 26
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
- 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
14 more
- 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
- 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 274
- 2023An Overview of Catastrophic AI Riskscited by 222
- 2025Real-Time Execution of Action Chunking Flow Policiescited by 216
- 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 128
- 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonablecited by 122
- 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
- 2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 63
- 2024Multimodal Situational Safetycited by 58
- 2025Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?cited by 38
- 2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 33
- 2025Safety Aware Task Planning via Large Language Models in Roboticscited by 24
- 2025Benchmarking the Robustness of Agentic Systems to Adversarially-Induced Harmscited by 10
- 2024Defining and Evaluating Physical Safety for Large Language Modelscited by 9
Built on by
Citing papers per year: 2025: 1; 2026: 13.
14 papers.
Sort:
By area:
- 2025Evaluating Gemini Robotics Policies in a Veo World Simulatorcited by 51
- 2026EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agentscited by 3
- 2026RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agentscited by 2
- 2026A Hazard-Informed Data Pipeline for Robotics Physical Safetycited by 1
- 2026HomeSafe-Bench: Evaluating Vision-Language Models on Unsafe Action Detection for Embodied Agents in Household Scenarioscited by 1
- 2026SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Modelscited by 1
- 2026ICAT: Incident-Case-Grounded Adaptive Testing for Physical-Risk Prediction in Embodied World Modelscited by 0
- 2026Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaborationcited by 0
- 2026VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoringcited by 0
- 2026Towards Trustworthy Physical Intelligence: From Theory to Practice Across Life Cyclecited by 0
- 2026No Free Checker: A Survey of Verifiers for Robot Policiescited by 0
- 2026ARC: Autonomous Robotics Compliance A Three-Layer Governance Architecture for Deployed Autonomous Systemscited by 0
2 more
- 2026Visual Grounding Safety in Vision-Language Models5 Oct 2026
- 2026PlanGuard: A Guardrail for Multi-Step Plan Safety in Embodied Agents26 Sep 2026
- 2026ARC: Autonomous Robotics Compliance A Three-Layer Governance Architecture for Deployed Autonomous Systems11 Sep 2026
- 2026No Free Checker: A Survey of Verifiers for Robot Policies8 Sep 2026
- 2026Towards Trustworthy Physical Intelligence: From Theory to Practice Across Life Cycle24 Jul 2026
- 2026VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring2 Jun 2026
- 2026Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaboration29 May 2026
- 2026EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents29 May 2026
- 2026RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents19 May 2026
- 2026SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models21 Apr 2026
- 2026ICAT: Incident-Case-Grounded Adaptive Testing for Physical-Risk Prediction in Embodied World Models31 Mar 2026
- 2026HomeSafe-Bench: Evaluating Vision-Language Models on Unsafe Action Detection for Embodied Agents in Household Scenarios12 Mar 2026
2 more
- 2026RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agentscited by 2
- 2026HomeSafe-Bench: Evaluating Vision-Language Models on Unsafe Action Detection for Embodied Agents in Household Scenarioscited by 1
- 2026SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Modelscited by 1
- 2026Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaborationcited by 0
- 2026Visual Grounding Safety in Vision-Language Modelscited by 0
- 2026Visual Grounding Safety in Vision-Language Models5 Oct 2026
- 2026Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaboration29 May 2026
- 2026RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents19 May 2026
- 2026SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models21 Apr 2026
- 2026HomeSafe-Bench: Evaluating Vision-Language Models on Unsafe Action Detection for Embodied Agents in Household Scenarios12 Mar 2026
- 2026A Hazard-Informed Data Pipeline for Robotics Physical Safetycited by 1
- 2026VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoringcited by 0
- 2026Towards Trustworthy Physical Intelligence: From Theory to Practice Across Life Cyclecited by 0
- 2026No Free Checker: A Survey of Verifiers for Robot Policiescited by 0
- 2026ARC: Autonomous Robotics Compliance A Three-Layer Governance Architecture for Deployed Autonomous Systemscited by 0
- 2026ARC: Autonomous Robotics Compliance A Three-Layer Governance Architecture for Deployed Autonomous Systems11 Sep 2026
- 2026No Free Checker: A Survey of Verifiers for Robot Policies8 Sep 2026
- 2026Towards Trustworthy Physical Intelligence: From Theory to Practice Across Life Cycle24 Jul 2026
- 2026VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring2 Jun 2026
- 2026A Hazard-Informed Data Pipeline for Robotics Physical Safety6 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses
- 2025ANNIE: Be Careful of Your Robots
- 2024EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents
- 2026SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents
- 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
- 2026GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
- 2026Silent Failures in Physical AI: A Literature Review of Runtime Action Authorization for Autonomous Systems
- 2026HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task