mlsanityData to 7 Oct 2026

Red Teaming Visual Language Models

arXiv 2401.12915 ↗First posted 23 Jan 2024cs.AI cs.CL cs.CVcited by 62

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

VLMs (Vision-Language Models) extend the capabilities of LLMs (Large Language Models) to accept multimodal inputs. Since it has been verified that LLMs can be induced to generate harmful or inaccurate content through specific test cases (termed as Red Teaming), how VLMs perform in similar scenarios, especially with their combination of textual and visual inputs, remains a question. To explore this problem, we present a novel red teaming dataset RTVLM, which encompasses 10 subtasks (e.g., image misleading, multi-modal jail-breaking, face fairness, etc) under 4 primary aspects (faithfulness, privacy, safety, fairness). Our RTVLM is the first red-teaming dataset to benchmark current VLMs in terms of these 4 different aspects. Detailed analysis shows that 10 prominent open-sourced VLMs struggle with the red teaming in different degrees and have up to 31% performance gap with GPT-4V. Additionally, we simply apply red teaming alignment to LLaVA-v1.5 with Supervised Fine-tuning (SFT) using RTVLM, and this bolsters the models' performance with 10% in RTVLM test set, 13% in MM-Hal, and without noticeable decline in MM-Bench, overpassing other LLaVA-based models with regular alignment data. This reveals that current open-sourced VLMs still lack red teaming alignment. Our code and datasets will be open-source.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 48 references that are in the map. First posted on arXiv 23 Jan 2024.

Builds on

31 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Can Language Models be Instructed to Protect Personal Information?3 Oct

Other references · 30

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  8. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  9. 2022Red Teaming Language Models with Language Modelscited by 983
  10. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  11. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  12. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
18 more
  1. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  2. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  3. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
  4. 2021True Few-Shot Learning with Language Modelscited by 343
  5. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  6. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 272
  7. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  8. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 225
  9. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  10. 2023MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learningcited by 171
  11. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  12. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  13. 2023Silkie: Preference Distillation for Large Visual Language Modelscited by 128
  14. 2023GPT-4V(ision) as a Generalist Evaluator for Vision-Language Taskscited by 118
  15. 2016A Hierarchical Approach for Generating Descriptive Image Paragraphscited by 104
  16. 2021Detecting Harmful Memes and Their Targetscited by 77
  17. 2024GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abusecited by 50
  18. 2022Knowledge Mining with Scene Text for Fine-Grained Recognitioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 25; 2025: 29; 2026: 8.

62 papers.

Sort:

By area:

  1. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  2. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  3. 2024An Introduction to Vision-Language Modelingcited by 175
  4. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Modelscited by 121
  5. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  6. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113
  7. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 82
  8. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  9. 2025MM-RLHF: The Next Step Forward in Multimodal LLM Alignmentcited by 79
  10. 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
  11. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  12. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
38 more
  1. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 57
  2. 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 51
  3. 2024A Survey on Evaluation of Multimodal Large Language Modelscited by 46
  4. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  5. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42
  6. 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 32
  7. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Modelscited by 29
  8. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  9. 2024Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Modelscited by 23
  10. 2025MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environmentscited by 23
  11. 2025Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Modelscited by 21
  12. 2024The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defensecited by 20
  13. 2025HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluationcited by 19
  14. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Securitycited by 18
  15. 2024VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wildcited by 17
  16. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  17. 2024A Survey on Safe Multi-Modal Learning Systemcited by 15
  18. 2024Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs)cited by 15
  19. 2025RvLLM: LLM Runtime Verification with Domain Knowledgecited by 15
  20. 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 14
  21. 2025Learning to Steer: Input-dependent Steering for Multimodal LLMscited by 13
  22. 2025Aligning Multimodal LLM with Human Preference: A Surveycited by 12
  23. 2025VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimizationcited by 10
  24. 2025USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Modelscited by 10
  25. 2024EACO: Enhancing Alignment in Multimodal LLMs via Critical Observationcited by 8
  26. 2025PiCo: Jailbreaking Multimodal Large Language Models via Pictorial Code Contextualizationcited by 8
  27. 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Modelscited by 8
  28. 2024Defending LVLMs Against Vision Attacks through Partial-Perception Supervisioncited by 7
  29. 2025Spot Risks Before Speaking! Unraveling Safety Attention Heads in Large Vision-Language Modelscited by 7
  30. 2025DAVSP: Safety Alignment for Large Vision-Language Models via Deep Aligned Visual Safety Promptcited by 7
  31. 2025DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Modelscited by 6
  32. 2025ELITE: Enhanced Language-Image Toxicity Evaluation for Safetycited by 5
  33. 2025Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalitiescited by 4
  34. 2025Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigationcited by 4
  35. 2025Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spotscited by 3
  36. 2025Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gapcited by 3
  37. 2025ARMs: Adaptive Red-Teaming Agent against Multimodal Models with Plug-and-Play Attackscited by 3
  38. 2025Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Modelscited by 2

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts
  2. 2026REALM: A Unified Red-Teaming Benchmark for Physical-World VLMs
  3. 2024Safety Alignment for Vision Language Models
  4. 2026DataComp-VLM: Improved Open Datasets for Vision-Language Models
  5. 2024VHELM: A Holistic Evaluation of Vision Language Models
  6. 2025Resource Consumption Red-Teaming for Large Vision-Language Models
  7. 2025VLURes: Benchmarking VLM Visual and Linguistic Understanding in Low-Resource Languages
  8. 2025From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens