mlsanityData to 7 Oct 2026

Adversary-Aware DPO: Enhancing Safety Alignment in Vision Language Models via Adversarial Training

arXiv 2502.11455 ↗First posted 17 Feb 2025cs.CRcited by 12

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Safety alignment is critical in pre-training large language models (LLMs) to generate responses aligned with human values and refuse harmful queries. Unlike LLM, the current safety alignment of VLMs is often achieved with post-hoc safety fine-tuning. However, these methods are less effective to white-box attacks. To address this, we propose textitAdversary-aware DPO (ADPO), a novel training framework that explicitly considers adversarial. textitAdversary-aware DPO (ADPO) integrates adversarial training into DPO to enhance the safety alignment of VLMs under worst-case adversarial perturbations. textitADPO introduces two key components: (1) an adversarial-trained reference model that generates human-preferred responses under worst-case perturbations, and (2) an adversarial-aware DPO loss that generates winner-loser pairs accounting for adversarial distortions. By combining these innovations, textitADPO ensures that VLMs remain robust and reliable even in the presence of sophisticated jailbreak attacks. Extensive experiments demonstrate that textitADPO outperforms baselines in the safety alignment and general utility of VLMs.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 23 of its 38 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

23 of 38 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Don't Say No: Jailbreaking LLM by Suppressing Refusal25 Apr
  2. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 20

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  9. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  10. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  11. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  12. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
8 more
  1. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  2. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  3. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 208
  4. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  5. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  6. 2024Efficient Adversarial Training in LLMs with Continuous Attackscited by 104
  7. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysiscited by 61
  8. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 59

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 5.

12 papers.

  1. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
  2. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 9
  3. 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Modelscited by 8
  4. 2025Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spotscited by 3
  5. 2025Harmonious Parameter Adaptation in Continual Visual Instruction Tuning for Safety-Aligned MLLMscited by 3
  6. 2026WARD: Adversarially Robust Defense of Web Agents Against Prompt Injectionscited by 3
  7. 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2
  8. 2025Agentic Moderation: Multi-Agent Design for Safer Vision-Language Modelscited by 1
  9. 2026SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safetycited by 0
  10. 2026Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguardscited by 0
  11. 2026The Uncontrolled Variable: Vision-Language Model Refusal Responds to Image Presence in Ways Risk Cannot Explaincited by 0
  12. 2026SafeMol: Dual-Modality Safety Alignment for Molecular Multimodal Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AdPO: Enhancing the Adversarial Robustness of Large Vision-Language Models with Preference Optimization
  2. 2025Rethinking Deep Alignment Through The Lens Of Incomplete Learning
  3. 2025Safety Alignment Can Be Not Superficial With Explicit Safety Signals
  4. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time
  5. 2025DAVSP: Safety Alignment for Large Vision-Language Models via Deep Aligned Visual Safety Prompt
  6. 2026Seeing No Evil: Blinding Large Vision-Language Models to Safety Instructions via Adversarial Attention Hijacking
  7. 2025Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attack
  8. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models