# DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Models

arXiv: https://arxiv.org/abs/2504.18053 · HTML: https://mlsanity.com/paper/2504.18053/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 19 of its 80 references that are in the map.
- First posted on arXiv 25 Apr 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)
- LLM alignment / RLHF / preference optimisation / [Safety alignment and jailbreak attacks](https://mlsanity.com/alignment/safety-alignment-and-jailbreak-attacks/)
- LLM safety and security / [Vision-language and text-to-image safety](https://mlsanity.com/safety/vision-language-and-text-to-image-safety/)

## Builds on (most cited first, 19 of 19)

- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues](https://mlsanity.com/paper/2402.14762/) (2024)
- [Red Teaming Visual Language Models](https://mlsanity.com/paper/2401.12915/) (2024)
- [LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Models](https://mlsanity.com/paper/2406.05113/) (2024)
- [GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models](https://mlsanity.com/paper/2406.14550/) (2024)
- [RATT: A Thought Structure for Coherent and Correct LLM Reasoning](https://mlsanity.com/paper/2406.02746/) (2024)
- [TrojVLM: Backdoor Attack Against Vision Language Models](https://mlsanity.com/paper/2409.19232/) (2024)
- [Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks](https://mlsanity.com/paper/2402.00626/) (2024)
- [Dynamic and Adaptive Feature Generation with LLM](https://mlsanity.com/paper/2406.03505/) (2024)
- [Cross-Modal Safety Alignment: Is textual unlearning all you need?](https://mlsanity.com/paper/2406.02575/) (2024)
- [Task-Agnostic Detector for Insertion-Based Backdoor Attacks](https://mlsanity.com/paper/2403.17155/) (2024)
- [Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Level](https://mlsanity.com/paper/2406.11817/) (2024)
- [Privacy-Aware Visual Language Models](https://mlsanity.com/paper/2405.17423/) (2024)
- [ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models](https://mlsanity.com/paper/2402.14660/) (2024)
- [Blind Spot Navigation in Large Language Model Reasoning with Thought Space Explorer](https://mlsanity.com/paper/2410.24155/) (2024)
- [2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision](https://mlsanity.com/paper/2410.19720/) (2024)
- [Beyond Bounding Box: Multimodal Knowledge Learning for Object Detection](https://mlsanity.com/paper/2205.04072/) (2022)
- [Regression and Forecasting of U.S. Stock Returns Based on LSTM](https://mlsanity.com/paper/2502.05210/) (2025)

## Built on by (most cited first, 6 of 6)

- [MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents](https://mlsanity.com/paper/2508.13186/) (2025)
- [Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignment](https://mlsanity.com/paper/2602.10161/) (2026)
- [Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignment](https://mlsanity.com/paper/2510.13698/) (2025)
- [Two Birds, One Projection: Harmonizing Safety and Utility in LVLMs via Inference-time Feature Projection](https://mlsanity.com/paper/2603.14825/) (2026)
- [Erase to Retain: Low Rank Adaptation Guided Selective Unlearning in Medical Segmentation Networks](https://mlsanity.com/paper/2511.16574/) (2025)
- [Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignment](https://mlsanity.com/paper/2609.37837/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) pose unique safety challenges due to their integration of visual and textual data, thereby introducing new dimensions of potential attacks and complex risk combinations. In this paper, we begin with a detailed analysis aimed at disentangling risks through step-by-step reasoning within multimodal inputs. We find that systematic multimodal risk disentanglement substantially enhances the risk awareness of MLLMs. Via leveraging the strong discriminative abilities of multimodal risk disentanglement, we further introduce DREAM (textitDisentangling Risks to Enhance Safety Alignment in MLLMs), a novel approach that enhances safety alignment in MLLMs through supervised fine-tuning and iterative Reinforcement Learning from AI Feedback (RLAIF). Experimental results show that DREAM significantly boosts safety during both inference and training phases without compromising performance on normal tasks (namely oversafety), achieving a 16.17% improvement in the SIUO safe&effective score compared to GPT-4V. The data and code are available at https://github.com/Kizna1ver/DREAM.