mlsanityData to 7 Oct 2026

Safety Alignment for Vision Language Models

arXiv 2405.13581 ↗First posted 22 May 2024cs.CV cs.AIcited by 27

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 2 more
Abstract (arXiv)

Benefiting from the powerful capabilities of Large Language Models (LLMs), pre-trained visual encoder models connected to an LLMs can realize Vision Language Models (VLMs). However, existing research shows that the visual modality of VLMs is vulnerable, with attackers easily bypassing LLMs' safety alignment through visual modality features to launch attacks. To address this issue, we enhance the existing VLMs' visual modality safety alignment by adding safety modules, including a safety projector, safety tokens, and a safety head, through a two-stage training process, effectively improving the model's defense against risky images. For example, building upon the LLaVA-v1.5 model, we achieve a safety score of 8.26, surpassing the GPT-4V on the Red Teaming Visual Language Models (RTVLM) benchmark. Our method boasts ease of use, high flexibility, and strong controllability, and it enhances safety while having minimal impact on the model's general performance. Moreover, our alignment strategy also uncovers some possible risky content within commonly used open-source multimodal datasets. Our code will be open sourced after the anonymous review.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 39 references that are in the map. First posted on arXiv 22 May 2024.

Builds on

32 of 39 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images20 Jan
  2. 2024Image Safeguarding: Reasoning with Conditional Vision Language Model and Obfuscating Unsafe Content Counterfactually19 Jan
  3. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov
  4. 2023AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectors26 Oct
  5. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  9. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  11. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  12. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
15 more
  1. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  2. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  3. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  4. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  5. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  6. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  7. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 225
  8. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 195
  9. 2023OneLLM: One Framework to Align All Modalities with Languagecited by 185
  10. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
  11. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  12. 2016A Hierarchical Approach for Generating Descriptive Image Paragraphscited by 104
  13. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 103
  14. 2023SEED-Bench-2: Benchmarking Multimodal Large Language Modelscited by 96
  15. 2023JailGuard: A Universal Detection Framework for LLM Prompt-based Attackscited by 46

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 16; 2026: 7.

27 papers.

Sort:

By area:

  1. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  2. 2025Safety in Large Reasoning Models: A Surveycited by 58
  3. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
  4. 2025BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimizationcited by 37
  5. 2024AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Drivingcited by 35
  6. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  7. 2024Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Modelscited by 23
  8. 2025JailBound: Jailbreaking Internal Safety Boundaries of Vision-Language Modelscited by 8
  9. 2025DAVSP: Safety Alignment for Large Vision-Language Models via Deep Aligned Visual Safety Promptcited by 7
  10. 2025Hate in Plain Sight: On the Risks of Moderating AI-Generated Hateful Illusionscited by 6
  11. 2025Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalitiescited by 4
  12. 2025GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervisioncited by 4
15 more
  1. 2026Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignmentcited by 4
  2. 2024Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Modelscited by 3
  3. 2025Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spotscited by 3
  4. 2025SafeCoT: Improving VLM Safety with Minimal Reasoningcited by 3
  5. 2026Steering to Say No: Configurable Refusal via Activation Steering in Vision Language Modelscited by 3
  6. 2026WARD: Adversarially Robust Defense of Web Agents Against Prompt Injectionscited by 3
  7. 2025From Evaluation to Defense: Advancing Safety in Video Large Language Modelscited by 2
  8. 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2
  9. 2025Enhancing the Safety of Medical Vision-Language Models by Synthetic Demonstrationscited by 1
  10. 2025LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Modelscited by 1
  11. 2026One-shot Optimized Steering Vector for Hallucination Mitigation for VLMscited by 1
  12. 2025Defeating Cerberus: Concept-Guided Privacy-Leakage Mitigation in Multimodal Language Modelscited by 0
  13. 2026TrustVLA: Mechanism-Guided Inference-Time Defense Against Vision-Language-Action Backdoorscited by 0
  14. 2026SafeRI: Recognition and Intervention for Token-Level Safety Intervention in Large Vision Language Modelscited by 0
  15. 2026Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024PSA-VLM: Enhancing Vision-Language Model Safety through Progressive Concept-Bottleneck-Driven Alignment
  2. 2025VLM-Guard: Safeguarding Vision-Language Models via Fulfilling Safety Alignment Gap
  3. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model
  4. 2025Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gap
  5. 2024Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models
  6. 2025Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?
  7. 2025VSCBench: Bridging the Gap in Vision-Language Model Safety Calibration
  8. 2025Understanding and Rectifying Safety Perception Distortion in VLMs