mlsanityData to 7 Oct 2026

Multimodal Situational Safety

arXiv 2410.06172 ↗First posted 8 Oct 2024cs.AI cs.CLcited by 54

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) are rapidly evolving, demonstrating impressive capabilities as multimodal assistants that interact with both humans and their environments. However, this increased sophistication introduces significant safety concerns. In this paper, we present the first evaluation and analysis of a novel safety challenge termed Multimodal Situational Safety, which explores how safety considerations vary based on the specific situation in which the user or agent is engaged. We argue that for an MLLM to respond safely, whether through language or action, it often needs to assess the safety implications of a language query within its corresponding visual context. To evaluate this capability, we develop the Multimodal Situational Safety benchmark (MSSBench) to assess the situational safety performance of current MLLMs. The dataset comprises 1,820 language query-image pairs, half of which the image context is safe, and the other half is unsafe. We also develop an evaluation framework that analyzes key safety aspects, including explicit safety reasoning, visual understanding, and, crucially, situational safety reasoning. Our findings reveal that current MLLMs struggle with this nuanced safety problem in the instruction-following setting and struggle to tackle these situational safety challenges all at once, highlighting a key area for future research. Furthermore, we develop multi-agent pipelines to coordinately solve safety challenges, which shows consistent improvement in safety over the original MLLM response. Code and data: mssbench.github.io.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. First posted on arXiv 8 Oct 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 32; 2026: 22.

54 papers.

Sort:

By area:

  1. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  2. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agentscited by 42
  3. 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Taskscited by 40
  4. 2025Safety Guardrails for LLM-Enabled Robotscited by 30
  5. 2025Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Modelscited by 21
  6. 2025Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Modelcited by 18
  7. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  8. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 15
  9. 2025Generative RLHF-V: Learning Principles from Multi-modal Human Preferencecited by 13
  10. 2025SafeMind: Benchmarking and Mitigating Safety Risks in Embodied LLM Agentscited by 13
  11. 2025Can AI Perceive Physical Danger and Intervene?cited by 12
  12. 2025USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Modelscited by 10
38 more
  1. 2025Praxis-VLM: Vision-Grounded Decision Making via Text-Driven Reinforcement Learningcited by 9
  2. 2025Multimodal Cultural Safety: Evaluation Framework and Alignment Strategiescited by 8
  3. 2025AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyondcited by 8
  4. 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Modelscited by 8
  5. 2025Towards Evaluating Proactive Risk Awareness of Multimodal Language Modelscited by 6
  6. 2025HoliSafe: Holistic Safety Benchmarking and Modeling for Vision-Language Modelcited by 6
  7. 2025Steering Multimodal Large Language Models Decoding for Context-Aware Safetycited by 6
  8. 2026Towards Policy-Adaptive Image Guardrail: Benchmark and Methodcited by 6
  9. 2025ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMscited by 5
  10. 2025Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attackcited by 5
  11. 2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5
  12. 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attackscited by 4
  13. 2026SAVeS: Steering Safety Judgments in Vision-Language Models via Semantic Cuescited by 4
  14. 2025Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failurescited by 3
  15. 2025SafeCoT: Improving VLM Safety with Minimal Reasoningcited by 3
  16. 2025SIA: Enhancing Safety via Intent Awareness for Vision-Language Modelscited by 3
  17. 2025Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systemscited by 3
  18. 2026ForesightSafety Bench: A Frontier Risk Evaluation and Governance Framework towards Safe AIcited by 3
  19. 2026Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMscited by 3
  20. 2026HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Taskcited by 3
  21. 2025Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoringcited by 2
  22. 2025PRISM: Robust VLM Alignment with Principled Reasoning for Integrated Safety in Multimodalitycited by 2
  23. 2025Falcon: A Cross-Modal Evaluation Dataset for Comprehensive Safety Perceptioncited by 2
  24. 2026The Side Effects of Being Smart: Safety Risks in MLLMs' Multi-Image Reasoningcited by 2
  25. 2025MDIT-Bench: Evaluating the Dual-Implicit Toxicity in Large Multimodal Modelscited by 1
  26. 2025Think-Reflect-Revise: A Policy-Guided Reflective Framework for Safety Alignment in Large Vision Language Modelscited by 1
  27. 2026If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systemscited by 1
  28. 2026MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Modelscited by 1
  29. 2025VIVA+: Human-Centered Situational Decision-Makingcited by 0
  30. 2025Visual Attention Reasoning via Hierarchical Search and Self-Verificationcited by 0
  31. 2026MM-SCALE: Grounded Multimodal Moral Reasoning via Scalar Judgment and Listwise Alignmentcited by 0
  32. 2026SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safetycited by 0
  33. 2026OOD-MMSafe: Advancing MLLM Safety from Harmful Intent to Hidden Consequencescited by 0
  34. 2026RobotValues: Evaluating Household Robots When Human Values Conflictcited by 0
  35. 2026Beyond Attack Success Rate: Examining Trigger Leakage in Vision-Language Agentic Systemscited by 0
  36. 2026Self-Evolving Just-In-Time Memory for Proactive Embodied Safetycited by 0
  37. 2026Multimodal Model Diffing for Feature Discovery and Controlcited by 0
  38. 2026Distributed Implicit Harm: A Compositional Safety Blind Spot in MLLM-Based Video Moderationcited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life
  2. 2025Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
  3. 2026MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialogues
  4. 2026Transfer Safety Awareness for Cross-Modal Safety Drift in Multimodal Large Language Models
  5. 2025MSTS: A Multimodal Safety Test Suite for Vision-Language Models
  6. 2026CSR-Bench: A Benchmark for Evaluating the Cross-modal Safety and Reliability of MLLMs
  7. 2025SafeMT: Multi-turn Safety for Multimodal Language Models
  8. 2026MLLMs Fail to Refuse when Using Tools Agentically