mlsanityData to 7 Oct 2026

CSR-Bench: A Benchmark for Evaluating the Cross-modal Safety and Reliability of MLLMs

arXiv 2602.03263 ↗First posted 3 Feb 2026cs.AIcited by 2

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Multimodal large language models (MLLMs) enable interaction over both text and images, but their safety behavior can be driven by unimodal shortcuts instead of true joint intent understanding. We introduce CSR-Bench, a benchmark for evaluating cross-modal reliability through four stress-testing interaction patterns spanning Safety, Over-rejection, Bias, and Hallucination, covering 61 fine-grained types. Each instance is constructed to require integrated image-text interpretation, and we additionally provide paired text-only controls to diagnose modality-induced behavior shifts. We evaluate 16 state-of-the-art MLLMs and observe systematic cross-modal alignment gaps. Models show weak safety awareness, strong language dominance under interference, and consistent performance degradation from text-only controls to multimodal inputs. We also observe a clear trade-off between reducing over-rejection and maintaining safe, non-discriminatory behavior, suggesting that some apparent safety gains may come from refusal-oriented heuristics rather than robust intent understanding. WARNING: This paper contains unsafe contents.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 29 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

11 of 29 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs16 Feb
  2. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safety29 Nov
  3. 2024MOSSBench: Is Your Multimodal Language Model Oversensitive to Safe Queries?22 Jun

Other references · 8

  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  2. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  3. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  4. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
  5. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 54
  6. 2024Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Educationcited by 15
  7. 2025MMDT: Decoding the Trustworthiness and Safety of Multimodal Foundation Modelscited by 14
  8. 2025Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimizationcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026EviSafe: Evidence-Grounded Safety Evaluation for Vision-Language Modelscited by 1
  2. 2026Constitutional On-Policy Safe Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Transfer Safety Awareness for Cross-Modal Safety Drift in Multimodal Large Language Models
  2. 2024Multimodal Situational Safety
  3. 2026MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialogues
  4. 2026Harnessing Textual Refusal Directions for Multimodal Safety
  5. 2026When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life
  6. 2026MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs
  7. 2025Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models
  8. 2026MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models