# MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Models

arXiv: https://arxiv.org/abs/2506.19257 · HTML: https://mlsanity.com/paper/2506.19257/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 21 of its 42 references that are in the map.
- First posted on arXiv 24 Jun 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)
- LLM safety and security / [Vision-language and text-to-image safety](https://mlsanity.com/safety/vision-language-and-text-to-image-safety/)

## Builds on (most cited first, 21 of 21)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models](https://mlsanity.com/paper/2311.06607/) (2023)
- [MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency](https://mlsanity.com/paper/2502.09621/) (2025)
- [STAR-1: Safer Alignment of Reasoning LLMs with 1K Data](https://mlsanity.com/paper/2504.01903/) (2025)
- [SPEED: Scalable, Precise, and Efficient Concept Erasure for Diffusion Models](https://mlsanity.com/paper/2503.07392/) (2025)
- [HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States](https://mlsanity.com/paper/2502.14744/) (2025)
- [MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs](https://mlsanity.com/paper/2505.21327/) (2025)
- [Separable Multi-Concept Erasure from Diffusion Models](https://mlsanity.com/paper/2402.05947/) (2024)
- [Equilibrate RLHF: Towards Balancing Helpfulness-Safety Trade-off in Large Language Models](https://mlsanity.com/paper/2502.11555/) (2025)
- [RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Prompting](https://mlsanity.com/paper/2412.18826/) (2024)
- [DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extraction](https://mlsanity.com/paper/2310.04567/) (2023)
- [Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025](https://mlsanity.com/paper/2506.12430/) (2025)
- [Dual Conic Proxies for AC Optimal Power Flow](https://mlsanity.com/paper/2310.02969/) (2023)
- [Fundamental Limits of Hierarchical Secure Aggregation with Cyclic User Association](https://mlsanity.com/paper/2503.04564/) (2025)
- [Self-Adaptive Gamma Context-Aware SSM-based Model for Metal Defect Detection](https://mlsanity.com/paper/2503.01234/) (2025)
- [A Critique of Human-Autonomous Team Dynamics: Contrasting Qualitative and Quantitative Perspectives](https://mlsanity.com/paper/2312.06789/) (2023)
- [Nonlinear and degenerate discounted approximation in discrete weak KAM theory](https://mlsanity.com/paper/2403.04563/) (2024)

## Built on by (most cited first, 7 of 7)

- [QuadSentinel: Sequent Safety for Machine-Checkable Control in Multi-agent Systems](https://mlsanity.com/paper/2512.16279/) (2025)
- [BLM-Guard: Explainable Multimodal Ad Moderation with Chain-of-Thought and Policy-Aligned Rewards](https://mlsanity.com/paper/2602.18193/) (2026)
- [From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Models](https://mlsanity.com/paper/2511.12861/) (2025)
- [Think-Reflect-Revise: A Policy-Guided Reflective Framework for Safety Alignment in Large Vision Language Models](https://mlsanity.com/paper/2512.07141/) (2025)
- [PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrails](https://mlsanity.com/paper/2607.05910/) (2026)
- [V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure](https://mlsanity.com/paper/2607.21151/) (2026)
- [Taming CoT Obfuscation in VLMs: From Mechanistic Evidence to Activation Enforcement](https://mlsanity.com/paper/2609.24243/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) have achieved remarkable progress in multimodal reasoning tasks through enhanced chain-of-thought capabilities. However, this advancement also introduces novel safety risks, as these models become increasingly vulnerable to harmful multimodal prompts that can trigger unethical or unsafe behaviors. Existing safety alignment approaches, primarily designed for unimodal language models, fall short in addressing the complex and nuanced threats posed by multimodal inputs. Moreover, current safety datasets lack the fine-grained, policy-grounded reasoning required to robustly align reasoning-capable VLMs. In this work, we introduce {MSR-Align}, a high-quality Multimodal Safety Reasoning dataset tailored to bridge this gap. MSR-Align supports fine-grained, deliberative reasoning over standardized safety policies across both vision and text modalities. Our data generation pipeline emphasizes multimodal diversity, policy-grounded reasoning, and rigorous quality filtering using strong multimodal judges. Extensive experiments demonstrate that fine-tuning VLMs on MSR-Align substantially improves robustness against both textual and vision-language jailbreak attacks, while preserving or enhancing general reasoning performance. MSR-Align provides a scalable and effective foundation for advancing the safety alignment of reasoning-capable VLMs. Our dataset is made publicly available at https://huggingface.co/datasets/Leigest/MSR-Align.