mlsanityData to 14 Oct 2026

MOSSBench: Is Your Multimodal Language Model Oversensitive to Safe Queries?

arXiv 2406.17806 ↗First posted 22 Jun 2024cs.CL cs.AI cs.CRcited by 30

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Humans are prone to cognitive distortions -- biased thinking patterns that lead to exaggerated responses to specific stimuli, albeit in very different contexts. This paper demonstrates that advanced Multimodal Large Language Models (MLLMs) exhibit similar tendencies. While these models are designed to respond queries under safety mechanism, they sometimes reject harmless queries in the presence of certain visual stimuli, disregarding the benign nature of their contexts. As the initial step in investigating this behavior, we identify three types of stimuli that trigger the oversensitivity of existing MLLMs: Exaggerated Risk, Negated Harm, and Counterintuitive Interpretation. To systematically evaluate MLLMs' oversensitivity to these stimuli, we propose the Multimodal OverSenSitivity Benchmark (MOSSBench). This toolkit consists of 300 manually collected benign multimodal queries, cross-verified by third-party reviewers (AMT). Empirical studies using MOSSBench on 20 MLLMs reveal several insights: (1). Oversensitivity is prevalent among SOTA MLLMs, with refusal rates reaching up to 76% for harmless queries. (2). Safer models are more oversensitive: increasing safety may inadvertently raise caution and conservatism in the model's responses. (3). Different types of stimuli tend to cause errors at specific stages -- perception, intent reasoning, and safety judgement -- in the response process of MLLMs. These findings highlight the need for refined safety mechanisms that balance caution with contextually appropriate responses, improving the reliability of MLLMs in real-world applications. We make our project available at https://turningpoint-ai.github.io/MOSSBench/.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 34 references that are in the map. First posted on arXiv 22 Jun 2024.

Builds on

13 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov

Other references · 12

  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2022Red Teaming Language Models with Language Modelscited by 1,026
  3. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  4. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  5. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 108
  6. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
  7. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 91
  8. 2023Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Modelscited by 60
  9. 2023LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?cited by 49
  10. 2023A Comprehensive Overview of Backdoor Attacks in Large Language Models within Communication Networkscited by 42
  11. 2021Detecting Harmful Content On Online Platforms: What Platforms Need Vs. Where Research Efforts Gocited by 33
  12. 2024How Far Are We from Intelligent Visual Deductive Reasoning?cited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 15; 2026: 12.

30 papers.

Sort:

By area:

  1. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
  2. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 67
  3. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 63
  4. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  5. 2025Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMscited by 23
  6. 2025Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Modelscited by 21
  7. 2025Aligning Multimodal LLM with Human Preference: A Surveycited by 13
  8. 2025SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimizationcited by 12
  9. 2025USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Modelscited by 10
  10. 2025Beyond Surface-Level Patterns: An Essence-Driven Defense Framework Against Jailbreak Attacks in LLMscited by 9
  11. 2025VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safetycited by 8
  12. 2025Steering Multimodal Large Language Models Decoding for Context-Aware Safetycited by 7
18 more
  1. 2025ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusalcited by 6
  2. 2025DUAL-Bench: Measuring Over-Refusal and Robustness in Vision-Language Modelscited by 6
  3. 2025VSCBench: Bridging the Gap in Vision-Language Model Safety Calibrationcited by 5
  4. 2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5
  5. 2025When Safe Unimodal Inputs Collide: Optimizing Reasoning Chains for Cross-Modal Safety in Multimodal Large Language Modelscited by 4
  6. 2026MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialoguescited by 4
  7. 2026When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Lifecited by 3
  8. 2026Auditing Multimodal LLM Raters: Central Tendency Bias in Clinical Ordinal Scoringcited by 3
  9. 2025How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigationcited by 2
  10. 2026CSR-Bench: A Benchmark for Evaluating the Cross-modal Safety and Reliability of MLLMscited by 2
  11. 2025Reimagining Safety Alignment with An Imagecited by 1
  12. 2026Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Modelscited by 0
  13. 2026Dictionary-Aligned Concept Control for Safeguarding Multimodal LLMscited by 0
  14. 2026AOR-Bench: Do Large Audio Language Models Over-Refuse Pseudo-Harmful Queries?cited by 0
  15. 2026A2Safe: Counterfactual Evidence-Aligned Adaptive Agent Collaboration for Safe and Effective Visual Question Answeringcited by 0
  16. 2026Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignmentcited by 0
  17. 2026False Floors: LLM Safety Routing Evaluations Break Under Distribution Shiftcited by 0
  18. 2026Visual Grounding Safety in Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Side Effects of Being Smart: Safety Risks in MLLMs' Multi-Image Reasoning
  2. 2024Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective
  3. 2026Uncertainty-Aware Decision Making in Multimodal Large Language Models
  4. 2025SafeMT: Multi-turn Safety for Multimodal Language Models
  5. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Models
  6. 2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models
  7. 2025Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundary
  8. 2024Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectly