# CSR-Bench: A Benchmark for Evaluating the Cross-modal Safety and Reliability of MLLMs

arXiv: https://arxiv.org/abs/2602.03263 · HTML: https://mlsanity.com/paper/2602.03263/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)

## Builds on (most cited first, 11 of 11)

- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [Evaluating and Mitigating Discrimination in Language Model Decisions](https://mlsanity.com/paper/2312.03689/) (2023)
- [VLSBench: Unveiling Visual Leakage in Multimodal Safety](https://mlsanity.com/paper/2411.19939/) (2024)
- [Look-Back: Implicit Visual Re-focusing in MLLM Reasoning](https://mlsanity.com/paper/2507.03019/) (2025)
- [MOSSBench: Is Your Multimodal Language Model Oversensitive to Safe Queries?](https://mlsanity.com/paper/2406.17806/) (2024)
- [Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs](https://mlsanity.com/paper/2502.11184/) (2025)
- [Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Education](https://mlsanity.com/paper/2401.00832/) (2024)
- [MMDT: Decoding the Trustworthiness and Safety of Multimodal Foundation Models](https://mlsanity.com/paper/2503.14827/) (2025)
- [Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization](https://mlsanity.com/paper/2503.17928/) (2025)

## Built on by (most cited first, 2 of 2)

- [Constitutional On-Policy Safe Distillation](https://mlsanity.com/paper/2606.03089/) (2026)
- [EviSafe: Evidence-Grounded Safety Evaluation for Vision-Language Models](https://mlsanity.com/paper/2608.23313/) (2026)

## Abstract (arXiv)

Multimodal large language models (MLLMs) enable interaction over both text and images, but their safety behavior can be driven by unimodal shortcuts instead of true joint intent understanding. We introduce CSR-Bench, a benchmark for evaluating cross-modal reliability through four stress-testing interaction patterns spanning Safety, Over-rejection, Bias, and Hallucination, covering 61 fine-grained types. Each instance is constructed to require integrated image-text interpretation, and we additionally provide paired text-only controls to diagnose modality-induced behavior shifts. We evaluate 16 state-of-the-art MLLMs and observe systematic cross-modal alignment gaps. Models show weak safety awareness, strong language dominance under interference, and consistent performance degradation from text-only controls to multimodal inputs. We also observe a clear trade-off between reducing over-rejection and maintaining safe, non-discriminatory behavior, suggesting that some apparent safety gains may come from refusal-oriented heuristics rather than robust intent understanding. WARNING: This paper contains unsafe contents.