mlsanityData to 14 Oct 2026

DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Models

arXiv 2504.18053 ↗First posted 25 Apr 2025cs.CL cs.CVcited by 6

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 2 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) pose unique safety challenges due to their integration of visual and textual data, thereby introducing new dimensions of potential attacks and complex risk combinations. In this paper, we begin with a detailed analysis aimed at disentangling risks through step-by-step reasoning within multimodal inputs. We find that systematic multimodal risk disentanglement substantially enhances the risk awareness of MLLMs. Via leveraging the strong discriminative abilities of multimodal risk disentanglement, we further introduce DREAM (textitDisentangling Risks to Enhance Safety Alignment in MLLMs), a novel approach that enhances safety alignment in MLLMs through supervised fine-tuning and iterative Reinforcement Learning from AI Feedback (RLAIF). Experimental results show that DREAM significantly boosts safety during both inference and training phases without compromising performance on normal tasks (namely oversafety), achieving a 16.17% improvement in the SIUO safe&effective score compared to GPT-4V. The data and code are available at https://github.com/Kizna1ver/DREAM.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 80 references that are in the map. First posted on arXiv 25 Apr 2025.

Builds on

19 of 80 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024TrojVLM: Backdoor Attack Against Vision Language Models28 Sep
  2. 2024Cross-Modal Safety Alignment: Is textual unlearning all you need?27 May
  3. 2024Privacy-Aware Visual Language Models27 May
  4. 2024Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks1 Feb
  5. 2024Red Teaming Visual Language Models23 Jan

Other references · 14

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  3. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 250
  4. 2024LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Modelscited by 52
  5. 2024GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Modelscited by 43
  6. 2024RATT: A Thought Structure for Coherent and Correct LLM Reasoningcited by 41
  7. 2024Dynamic and Adaptive Feature Generation with LLMcited by 27
  8. 2024Task-Agnostic Detector for Insertion-Based Backdoor Attackscited by 23
  9. 2024Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Levelcited by 22
  10. 2024ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Modelscited by 10
  11. 2024Blind Spot Navigation in Large Language Model Reasoning with Thought Space Explorercited by 7
  12. 20242D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervisioncited by 6
2 more
  1. 2022Beyond Bounding Box: Multimodal Knowledge Learning for Object Detectioncited by 5
  2. 2025Regression and Forecasting of U.S. Stock Returns Based on LSTMcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agentscited by 42
  2. 2026Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignmentcited by 6
  3. 2025Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignmentcited by 4
  4. 2026Two Birds, One Projection: Harmonizing Safety and Utility in LVLMs via Inference-time Feature Projectioncited by 1
  5. 2025Erase to Retain: Low Rank Adaptation Guided Selective Unlearning in Medical Segmentation Networkscited by 0
  6. 2026Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguards
  2. 2026Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMs
  3. 2025When Safe Unimodal Inputs Collide: Optimizing Reasoning Chains for Cross-Modal Safety in Multimodal Large Language Models
  4. 2026ReFrame: Evidence-Guided Test-Time Safety Alignment in Multimodal Large Language Models
  5. 2025Automating Steering for Safe Multimodal Large Language Models
  6. 2025SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimization
  7. 2026SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models
  8. 2026OOD-MMSafe: Advancing MLLM Safety from Harmful Intent to Hidden Consequences