mlsanityData to 14 Oct 2026

Can AI Perceive Physical Danger and Intervene?

arXiv 2509.21651 ↗First posted 25 Sep 2025cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

When AI interacts with the physical world -- as a robot or an assistive agent -- new safety challenges emerge beyond those of purely ``digital AI". In such interactions, the potential for physical harm is direct and immediate. How well do state-of-the-art foundation models understand common-sense facts about physical safety, e.g. that a box may be too heavy to lift, or that a hot cup of coffee should not be handed to a child? In this paper, our contributions are three-fold: first, we develop a highly scalable approach to continuous physical safety benchmarking of Embodied AI systems, grounded in real-world injury narratives and operational safety constraints. To probe multi-modal safety understanding, we turn these narratives and constraints into photorealistic images and videos capturing transitions from safe to unsafe states, using advanced generative models. Secondly, we comprehensively analyze the ability of major foundation models to perceive risks, reason about safety, and trigger interventions; this yields multi-faceted insights into their deployment readiness for safety-critical agentic applications. Finally, we develop a post-training paradigm to teach models to explicitly reason about embodiment-specific safety constraints provided through system instructions. The resulting models generate thinking traces that make safety reasoning interpretable and transparent, achieving state of the art performance in constraint satisfaction evaluations. The benchmark is released at https://asimov-benchmark.github.io/v2

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 25 Sep 2025.

Builds on

26 of 42 references.

References · 26

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  8. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  9. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  10. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  11. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  12. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
14 more
  1. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  2. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 274
  3. 2023An Overview of Catastrophic AI Riskscited by 222
  4. 2025Real-Time Execution of Action Chunking Flow Policiescited by 216
  5. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 128
  6. 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonablecited by 122
  7. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
  8. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 63
  9. 2024Multimodal Situational Safetycited by 58
  10. 2025Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?cited by 38
  11. 2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 33
  12. 2025Safety Aware Task Planning via Large Language Models in Roboticscited by 24
  13. 2025Benchmarking the Robustness of Agentic Systems to Adversarially-Induced Harmscited by 10
  14. 2024Defining and Evaluating Physical Safety for Large Language Modelscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 13.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses
  2. 2025ANNIE: Be Careful of Your Robots
  3. 2024EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents
  4. 2026SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents
  5. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
  6. 2026GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
  7. 2026Silent Failures in Physical AI: A Literature Review of Runtime Action Authorization for Autonomous Systems
  8. 2026HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task