mlsanityData to 14 Oct 2026

SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model

arXiv 2406.12030 ↗First posted 17 Jun 2024cs.CV cs.AI cs.CLcited by 85

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 2 more
Abstract (arXiv)

The emergence of Vision Language Models (VLMs) has brought unprecedented advances in understanding multimodal information. The combination of textual and visual semantics in VLMs is highly complex and diverse, making the safety alignment of these models challenging. Furthermore, due to the limited study on the safety alignment of VLMs, there is a lack of large-scale, high-quality datasets. To address these limitations, we propose a Safety Preference Alignment dataset for Vision Language Models named SPA-VL. In terms of breadth, SPA-VL covers 6 harmfulness domains, 13 categories, and 53 subcategories, and contains 100,788 samples of the quadruple (question, image, chosen response, rejected response). In terms of depth, the responses are collected from 12 open-source (e.g., QwenVL) and closed-source (e.g., Gemini) VLMs to ensure diversity. The construction of preference data is fully automated, and the experimental results indicate that models trained with alignment techniques on the SPA-VL dataset exhibit substantial improvements in harmlessness and helpfulness while maintaining core capabilities. SPA-VL, as a large-scale, high-quality, and diverse dataset, represents a significant milestone in ensuring that VLMs achieve both harmlessness and helpfulness.

Key facts. Cited by 85 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 69 of its 85 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

69 of 85 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks3 Apr
  2. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models14 Mar
  3. 2024VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models21 Feb
  4. 2024Red Teaming Visual Language Models23 Jan
  5. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance5 Jan
  6. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov
  7. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  8. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  9. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
  10. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 59

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  11. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
38 more
  1. 2023A Survey of Large Language Modelscited by 2,576
  2. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  3. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  6. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  7. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  8. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  9. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  10. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  11. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  12. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  13. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,095
  14. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  15. 2021Ethical and social risks of harm from Language Modelscited by 966
  16. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  17. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  18. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  19. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  20. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  21. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  22. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  23. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 406
  24. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 400
  25. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  26. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 339
  27. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  28. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 285
  29. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
  30. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 198
  31. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 172
  32. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  33. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 167
  34. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  35. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 135
  36. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 108
  37. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 103
  38. 2024InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidancecited by 83

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 13; 2025: 43; 2026: 29.

85 papers.

Sort:

By area:

  1. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 239
  2. 2024Building and better understanding vision-language models: insights and future directionscited by 173
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  4. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 67
  5. 2025Safety in Large Reasoning Models: A Surveycited by 64
  6. 2024Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detectioncited by 63
  7. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 60
  8. 2024Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoningcited by 50
  9. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 42
  10. 2024SOLO: A Single Transformer for Scalable Vision-Language Modelingcited by 37
  11. 2024Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Modelscited by 37
  12. 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 34
38 more
  1. 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedbackcited by 30
  2. 2024A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applicationscited by 27
  3. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 23
  4. 2025Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Modelscited by 21
  5. 2024Jigsaw Puzzles: Splitting Harmful Questions to Jailbreak Large Language Modelscited by 19
  6. 2025Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Modelcited by 19
  7. 2025Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Modelscited by 18
  8. 2025Understanding and Rectifying Safety Perception Distortion in VLMscited by 17
  9. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  10. 2024Have the VLMs Lost Confidence? A Study of Sycophancy in VLMscited by 15
  11. 2025GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinkingcited by 15
  12. 2025SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimizationcited by 12
  13. 2025SEA: Low-Resource Safety Alignment for Multimodal Large Language Models via Synthetic Embeddingscited by 11
  14. 2025VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimizationcited by 10
  15. 2025MORALISE: A Structured Benchmark for Moral Alignment in Visual Language Modelscited by 10
  16. 2025USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Modelscited by 10
  17. 2025Towards Understanding Visual Grounding in Visual Language Modelscited by 10
  18. 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Modelscited by 10
  19. 2026SafeNeuron: Neuron-Level Safety Alignment for Large Language Modelscited by 10
  20. 2025HoliSafe: Holistic Safety Benchmarking and Modeling for Vision-Language Modelcited by 9
  21. 2024How Does Vision-Language Adaptation Impact the Safety of Vision Language Models?cited by 8
  22. 2025DAVSP: Safety Alignment for Large Vision-Language Models via Deep Aligned Visual Safety Promptcited by 8
  23. 2025SARSteer: Safeguarding Large Audio-Language Models via Safe-Ablated Refusal Steeringcited by 8
  24. 2025ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMscited by 7
  25. 2026Toward Native Multimodal Modeling: A Roadmapcited by 7
  26. 2025GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMscited by 6
  27. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 6
  28. 2026Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignmentcited by 6
  29. 2025Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spotscited by 4
  30. 2025InterMT: Multi-Turn Interleaved Preference Alignment with Human Feedbackcited by 4
  31. 2025Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalitiescited by 4
  32. 2025Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigationcited by 4
  33. 2025Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignmentcited by 4
  34. 2025ProGuard: Towards Proactive Multimodal Safeguardcited by 4
  35. 2024textitMMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Modelscited by 3
  36. 2025Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gapcited by 3
  37. 2025SIA: Enhancing Safety via Intent Awareness for Vision-Language Modelscited by 3
  38. 2025Harmonious Parameter Adaptation in Continual Visual Instruction Tuning for Safety-Aligned MLLMscited by 3

50 most cited of 85 All 85 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models
  2. 2025MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Models
  3. 2024Safety Alignment for Vision Language Models
  4. 2025VSCBench: Bridging the Gap in Vision-Language Model Safety Calibration
  5. 2024PSA-VLM: Enhancing Vision-Language Model Safety through Progressive Concept-Bottleneck-Driven Alignment
  6. 2025MSTS: A Multimodal Safety Test Suite for Vision-Language Models
  7. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safety
  8. 2025VLM-Guard: Safeguarding Vision-Language Models via Fulfilling Safety Alignment Gap