mlsanityData to 7 Oct 2026

AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions

arXiv 2506.14697 ↗First posted 17 Jun 2025cs.CR cs.ROcited by 33

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

The integration of vision-language models (VLMs) is driving a new generation of embodied agents capable of operating in human-centered environments. However, as deployment expands, these systems face growing safety risks, particularly when executing hazardous instructions. Current safety evaluation benchmarks remain limited: they cover only narrow scopes of hazards and focus primarily on final outcomes, neglecting the agent's full perception-planning-execution process and thereby obscuring critical failure modes. Therefore, we present SAFE, a benchmark for systematically assessing the safety of embodied VLM agents on hazardous instructions. SAFE comprises three components: SAFE-THOR, an extensible adversarial simulation sandbox with a universal adapter that maps high-level VLM outputs to low-level embodied controls, supporting diverse agent workflow integration; SAFE-VERSE, a risk-aware task suite inspired by Asimov's Three Laws of Robotics, comprising 45 adversarial scenarios, 1,350 hazardous tasks, and 9,900 instructions that span risks to humans, environments, and agents; and SAFE-DIAGNOSE, a multi-level and fine-grained evaluation protocol measuring agent performance across perception, planning, and execution. Applying SAFE to nine state-of-the-art VLMs and two embodied agent workflows, we uncover systematic failures in translating hazard recognition into safe planning and execution. Our findings reveal fundamental limitations in current safety alignment and demonstrate the necessity of a comprehensive, multi-stage evaluation for developing safer embodied intelligence.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 66 of its 91 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

66 of 91 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought20 Jun
  2. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation24 Dec
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 63

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  10. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  11. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  12. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
38 more
  1. 2023Large Language Modelscited by 774
  2. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  3. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  4. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  5. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  6. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
  7. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  8. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 254
  9. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  10. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  11. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 224
  12. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 222
  13. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 196
  14. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  15. 2025Multi-agent Architecture Search via Agentic Supernetcited by 164
  16. 2024Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Makingcited by 142
  17. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
  18. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  19. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 103
  20. 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agentscited by 102
  21. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 93
  22. 2024Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systemscited by 79
  23. 2024On the Role of Attention Heads in Large Language Model Safetycited by 70
  24. 2022SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehiclescited by 65
  25. 2023Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorldcited by 65
  26. 2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Modelscited by 63
  27. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Modelscited by 61
  28. 2019Training Robust Deep Neural Networks via Adversarial Noise Propagationcited by 60
  29. 2023X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detectioncited by 58
  30. 2025Safety in Large Reasoning Models: A Surveycited by 58
  31. 2025G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systemscited by 56
  32. 2024Semantic Mirror Jailbreak: Genetic Algorithm Based Jailbreak Prompts Against Open-source LLMscited by 48
  33. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 42
  34. 2021Harnessing Perceptual Adversarial Patches for Crowd Countingcited by 39
  35. 2023Pre-trained Trojan Attacks for Visual Recognitioncited by 38
  36. 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 35
  37. 2024Exploring the Robustness of Decision-Level Through Adversarial Attacks on LLM-Based Embodied Modelscited by 31
  38. 2024GenRL: Multimodal-foundation world models for generalization in embodied agentscited by 29

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 25.

33 papers.

Sort:

By area:

  1. 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Taskscited by 40
  2. 2025Graph-Augmented Large Language Model Agents: Current Progress and Future Prospectscited by 14
  3. 2025SafeMind: Benchmarking and Mitigating Safety Risks in Embodied LLM Agentscited by 13
  4. 2026Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanismscited by 12
  5. 2025Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editingcited by 9
  6. 2025Mask-GCG: Are All Tokens in Adversarial Suffixes Necessary for Jailbreak Attacks?cited by 6
  7. 2026HazardArena: Evaluating Semantic Safety in Vision-Language-Action Modelscited by 6
  8. 2025HomeSafeBench: Benchmarking Embodied Vision-Language Models in Free-Exploration Home Safety Inspectioncited by 4
  9. 2026SAVeS: Steering Safety Judgments in Vision-Language Models via Semantic Cuescited by 4
  10. 2026Using large language models for embodied planning introduces systematic safety riskscited by 4
  11. 2026HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Taskcited by 3
  12. 2025RoboView-Bias: Benchmarking Visual Bias in Embodied Agents for Robotic Manipulationcited by 2
21 more
  1. 2025Sequential Comics for Jailbreaking Multimodal Large Language Models via Structured Visual Storytellingcited by 2
  2. 2026LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agentscited by 2
  3. 2026Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluationcited by 2
  4. 2026Evolving Deception: When Agents Evolve, Deception Winscited by 1
  5. 2026EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systemscited by 1
  6. 2026Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agentscited by 1
  7. 2026Do Safety Guardrails Need to Reason? LeanGuard: A Fast and Light Approach for Robust Moderationcited by 1
  8. 2026Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultationcited by 0
  9. 2026A Patch-based Cross-view Regularized Framework for Backdoor Defense in Multimodal Large Language Modelscited by 0
  10. 2026GuardAD: Safeguarding Autonomous Driving MLLMs via Markovian Safety Logiccited by 0
  11. 2026Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaborationcited by 0
  12. 2026EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guardscited by 0
  13. 2026When Words Are Safe But Actions Kill: Probing Physical Jailbreak Beyond Textual Jailbreak in Hidden-State Risk Spacecited by 0
  14. 2026Self-Evolving Just-In-Time Memory for Proactive Embodied Safetycited by 0
  15. 2026Agents in the Wild: Where Research Meets Deploymentcited by 0
  16. 2026Breadcrumbing Search Agentscited by 0
  17. 2026A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applicationscited by 0
  18. 2026Beyond What Meets the Eye: Unveiling Situational Illusions for Multimodal Large Language Modelscited by 0
  19. 2026FailureSpot: Label-Efficient Timestamp-Level Failure Detection for Vision-Language-Action Modelscited by 0
  20. 2026Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directionscited by 0
  21. 2026TempQ-Jail: Query-Constrained Candidate Ranking for Text-to-Video Jailbreak Attackscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models
  2. 2026GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
  3. 2026RT-Safe: Benchmarking Agent Safety in Real-Time Embodied Environment
  4. 2026Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses
  5. 2026BeSafe-Bench: Unveiling Behavioral Safety Risks of Situated Agents in Functional Environments
  6. 2025ANNIE: Be Careful of Your Robots
  7. 2026SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents
  8. 2026SafeBranch: Branch-Pair Safety Alignment for Embodied Agents