mlsanityData to 14 Oct 2026

Towards Harmless Multimodal Assistants with Blind Preference Optimization

arXiv 2503.14189 ↗First posted 18 Mar 2025cs.CL cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in multimodal understanding, reasoning, and interaction. Given the extensive applications of MLLMs, the associated safety issues have become increasingly critical. Due to the effectiveness of preference optimization in aligning MLLMs with human preferences, there is an urgent need for safety-related preference data for MLLMs. To address this, we construct the MMSafe-PO preference dataset towards harmless multimodal assistants, featuring multimodal instructions, the conversational format, and ranked paired responses from human feedback. We also identify two insightful observations: modality co-defense and modality cheating, which illustrate that MLLMs possess a certain level of inherent defense while still presenting unique safety challenges. Based on these observations, we propose the Blind Preference Optimization (BPO) approach. Comprehensive experiments on three benchmarks show that BPO effectively enhances the safety capabilities of MLLMs. Notably, BPO significantly improves the safety rate of the base MLLM by 45.0%, outperforming the DPO approach. Additionally, applying BPO to the MMSafe-PO dataset greatly reduces the base MLLM's unsafe rate on other safety benchmarks (14.5% on MM-SafetyBench and 82.9% on HarmEval, demonstrating the effectiveness and robustness of both the dataset and the approach. We release code and data at https://lu-yang666.github.io/MMsafe-PO-Web/.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 6 of its 8 references that are in the map. First posted on arXiv 18 Mar 2025.

Builds on

6 of 8 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun

Other references · 5

  1. 2023LIMA: Less Is More for Alignmentcited by 1,100
  2. 2023VIGC: Visual Instruction Generation and Correctioncited by 80
  3. 2024VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignmentcited by 58
  4. 2024Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyondcited by 28
  5. 2024Revolutionizing Text-to-Image Retrieval as Autoregressive Token-to-Voken Generationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMscited by 3
  2. 2026OOD-MMSafe: Advancing MLLM Safety from Harmful Intent to Hidden Consequencescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
  2. 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedback
  3. 2026MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs
  4. 2024Multimodal Situational Safety
  5. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization
  6. 2025When Safe Unimodal Inputs Collide: Optimizing Reasoning Chains for Cross-Modal Safety in Multimodal Large Language Models
  7. 2026CSR-Bench: A Benchmark for Evaluating the Cross-modal Safety and Reliability of MLLMs
  8. 2026When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life