mlsanityData to 7 Oct 2026

Towards Evaluating Proactive Risk Awareness of Multimodal Language Models

arXiv 2505.17455 ↗First posted 23 May 2025cs.CL cs.AIcited by 6

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Human safety awareness gaps often prevent the timely recognition of everyday risks. In solving this problem, a proactive safety artificial intelligence (AI) system would work better than a reactive one. Instead of just reacting to users' questions, it would actively watch people's behavior and their environment to detect potential dangers in advance. Our Proactive Safety Bench (PaSBench) evaluates this capability through 416 multimodal scenarios (128 image sequences, 288 text logs) spanning 5 safety-critical domains. Evaluation of 36 advanced models reveals fundamental limitations: Top performers like Gemini-2.5-pro achieve 71% image and 64% text accuracy, but miss 45-55% risks in repeated trials. Through failure analysis, we identify unstable proactive reasoning rather than knowledge deficits as the primary limitation. This work establishes (1) a proactive safety benchmark, (2) systematic evidence of model limitations, and (3) critical directions for developing reliable protective AI. We believe our dataset and findings can promote the development of safer AI assistants that actively prevent harm rather than merely respond to requests. Our dataset can be found at https://huggingface.co/datasets/Youliang/PaSBench.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 46 of its 72 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

46 of 72 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Multimodal Situational Safety8 Oct
  2. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov

Other references · 44

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  6. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  7. 2022Red Teaming Language Models with Language Modelscited by 983
  8. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  9. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  10. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  11. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
  12. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
32 more
  1. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  2. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  3. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 406
  4. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  5. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
  6. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  7. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 280
  8. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
  9. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 211
  10. 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaborationcited by 115
  11. 2024STaR-GATE: Teaching Language Models to Ask Clarifying Questionscited by 99
  12. 2023ProAgent: Building Proactive Cooperative Agents with Large Language Modelscited by 97
  13. 2025SafeArena: Evaluating the Safety of Autonomous Web Agentscited by 97
  14. 2023Combating Misinformation in the Age of LLMs: Opportunities and Challengescited by 94
  15. 2024RedCode: Risky Code Execution and Generation Benchmark for Code Agentscited by 87
  16. 2024Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistancecited by 84
  17. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 77
  18. 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 70
  19. 2023A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospectscited by 66
  20. 2024Refusal-Trained LLMs Are Easily Jailbroken As Browser Agentscited by 61
  21. 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 56
  22. 2024Towards Human-centered Proactive Conversational Agentscited by 48
  23. 2024CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Modelscited by 48
  24. 2022SafeText: A Benchmark for Exploring Physical Safety in Language Modelscited by 41
  25. 2023Knowledge-enhanced Mixed-initiative Dialogue System for Emotional Support Conversationscited by 41
  26. 2023DisasterResponseGPT: Large Language Models for Accelerated Plan of Action Development in Disaster Response Scenarioscited by 33
  27. 2022Incidents1M: a large-scale dataset of images with natural disasters, damage, and incidentscited by 25
  28. 2024LLM-Assisted Crisis Management: Building Advanced LLM Platforms for Effective Emergency Response and Public Collaborationcited by 20
  29. 2024SportQA: A Benchmark for Sports Understanding in Large Language Modelscited by 15
  30. 2024Beyond Words: On Large Language Models Actionability in Mission-Critical Risk Analysiscited by 9
  31. 2024Large language models can help boost food production, but be mindful of their riskscited by 5
  32. 2025RESPONSE: Benchmarking the Ability of Language Models to Undertake Commonsense Reasoning in Crisis Situationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Taskcited by 3
  2. 2026When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Lifecited by 2
  3. 2026PaSBench-Video: A Streaming Video Benchmark for Proactive Safety Warningcited by 1
  4. 2026Evaluating Proactive Risk Awareness of Large Language Modelscited by 0
  5. 2026From Sports to Safety: Benchmarking Proactive Risk Inference in MLLMscited by 0
  6. 2026PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safetycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models
  2. 2026SafePro: Evaluating the Safety of Professional-Level AI Agents
  3. 2025Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
  4. 2025MSTS: A Multimodal Safety Test Suite for Vision-Language Models
  5. 2025VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety
  6. 2026The Side Effects of Being Smart: Safety Risks in MLLMs' Multi-Image Reasoning
  7. 2026MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models
  8. 2025SIA: Enhancing Safety via Intent Awareness for Vision-Language Models