# The Side Effects of Being Smart: Safety Risks in MLLMs' Multi-Image Reasoning

arXiv: https://arxiv.org/abs/2601.14127 · HTML: https://mlsanity.com/paper/2601.14127/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 30 of its 58 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)

## Builds on (most cited first, 30 of 30)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning](https://mlsanity.com/paper/2507.01006/) (2025)
- [Kimi-VL Technical Report](https://mlsanity.com/paper/2504.07491/) (2025)
- [JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2404.03027/) (2024)
- [Grounded Chain-of-Thought for Multimodal Large Language Models](https://mlsanity.com/paper/2503.12799/) (2025)
- [Multimodal Situational Safety](https://mlsanity.com/paper/2410.06172/) (2024)
- [Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmark](https://mlsanity.com/paper/2407.13766/) (2024)
- [MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection](https://mlsanity.com/paper/2410.09453/) (2024)
- [Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Models](https://mlsanity.com/paper/2412.05934/) (2024)
- [What Prompts Don't Say: Understanding and Managing Underspecification in LLM Prompts](https://mlsanity.com/paper/2505.13360/) (2025)
- [Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Models](https://mlsanity.com/paper/2501.18533/) (2025)
- [Skywork-R1V3 Technical Report](https://mlsanity.com/paper/2507.06167/) (2025)
- [Chain-of-Thought Hijacking](https://mlsanity.com/paper/2510.26418/) (2025)
- [Cognitive Overload Attack:Prompt Injection for Long Context](https://mlsanity.com/paper/2410.11272/) (2024)
- [VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Models](https://mlsanity.com/paper/2505.19684/) (2025)
- [AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement](https://mlsanity.com/paper/2502.16776/) (2025)
- [LongSafety: Evaluating Long-Context Safety of Large Language Models](https://mlsanity.com/paper/2502.16971/) (2025)
- [MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding](https://mlsanity.com/paper/2508.15802/) (2025)
- [United Minds or Isolated Agents? Exploring Coordination of LLMs under Cognitive Load Theory](https://mlsanity.com/paper/2506.06843/) (2025)
- [ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMs](https://mlsanity.com/paper/2505.14035/) (2025)
- [Self-Attention Limits Working Memory Capacity of Transformer-Based Models](https://mlsanity.com/paper/2409.10715/) (2024)
- [ELITE: Enhanced Language-Image Toxicity Evaluation for Safety](https://mlsanity.com/paper/2502.04757/) (2025)

## Built on by (most cited first, 3 of 3)

- [Safe Alone, Unsafe Together: Safeguarding Against Implicit Toxicity When Benign Images Combine](https://mlsanity.com/paper/2607.00576/) (2026)
- [Automatic Hard Example Synthesis with Multi-Level Agentic Data Curation](https://mlsanity.com/paper/2607.14256/) (2026)
- [Scenes as an Attack Surface: Uncovering Scene-Grounded Safety Failures in MLLMs](https://mlsanity.com/paper/2610.14094/) (2026)

## Abstract (arXiv)

As Multimodal Large Language Models (MLLMs) acquire stronger reasoning capabilities to handle complex, multi-image instructions, this advancement may pose new safety risks. We study this problem by introducing MIR-SafetyBench, the first benchmark focused on multi-image reasoning safety, which consists of 2,676 instances across a taxonomy of 9 multi-image relations. Our extensive evaluations on 19 MLLMs reveal a troubling trend: models with more advanced multi-image reasoning can be more vulnerable on MIR-SafetyBench. Beyond attack success rates, we find that many responses labeled as safe are superficial, often driven by misunderstanding or evasive, non-committal replies. We further observe that unsafe generations exhibit lower attention entropy than safe ones on average. This internal signature suggests a possible risk that models may over-focus on task solving while neglecting safety constraints. Our code and data are available at https://github.com/thu-coai/MIR-SafetyBench.