mlsanityData to 7 Oct 2026

SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models

arXiv 2604.19638 ↗First posted 21 Apr 2026cs.AI cs.CL cs.ROcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Multimodal Large Language Models are increasingly adopted as autonomous agents in interactive environments, yet their ability to proactively address safety hazards remains insufficient. We introduce SafetyALFRED, built upon the embodied agent benchmark ALFRED, augmented with six categories of real-world kitchen hazards. While existing safety evaluations focus on hazard recognition through disembodied question answering (QA) settings, we evaluate eleven state-of-the-art models from the Qwen, Gemma, and Gemini families on not only hazard recognition, but also active risk mitigation through embodied planning. Our experimental results reveal a significant alignment gap: while models can accurately recognize hazards in QA settings, average mitigation success rates for these hazards are low in comparison. Our findings demonstrate that static evaluations through QA are insufficient for physical safety, thus we advocate for a paradigm shift toward benchmarks that prioritize corrective actions in embodied contexts. We open-source our code and dataset under https://github.com/sled-group/SafetyALFRED.git

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 39 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

23 of 39 references.

References · 23

  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  5. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  6. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  7. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  8. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  9. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  10. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  11. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 252
  12. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
11 more
  1. 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agentscited by 102
  2. 2023Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agentscited by 67
  3. 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Taskscited by 40
  4. 2024HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environmentscited by 32
  5. 2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 28
  6. 2025A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agentscited by 27
  7. 2025Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Makingcited by 19
  8. 2025SafeMind: Benchmarking and Mitigating Safety Risks in Embodied LLM Agentscited by 13
  9. 2025Can AI Perceive Physical Danger and Intervene?cited by 12
  10. 2024Safe Planner: Empowering Safety Awareness in Large Pre-Trained Models for Robot Task Planningcited by 9
  11. 2024Safety Control of Service Robots with LLMs and Embodied Knowledge Graphscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Evaluating Proactive Risk Awareness of Multimodal Language Models
  2. 2026SafePro: Evaluating the Safety of Professional-Level AI Agents
  3. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
  4. 2024Multimodal Situational Safety
  5. 2026Using large language models for embodied planning introduces systematic safety risks
  6. 2026EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents
  7. 2026BeSafe-Bench: Unveiling Behavioral Safety Risks of Situated Agents in Functional Environments
  8. 2026ARGOS: Automated Functional Safety Requirement Synthesis for Embodied AI via Attribute-Guided Combinatorial Reasoning