mlsanityData to 14 Oct 2026

Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models

arXiv 2501.09039 ↗First posted 14 Jan 2025cs.CR cs.AI cs.CYcited by 2

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

The rapid advancement of Large Vision-Language Models (LVLMs) has enhanced capabilities offering potential applications from content creation to productivity enhancement. Despite their innovative potential, LVLMs exhibit vulnerabilities, especially in generating potentially toxic or unsafe responses. Malicious actors can exploit these vulnerabilities to propagate toxic content in an automated (or semi-) manner, leveraging the susceptibility of LVLMs to deception via strategically crafted prompts without fine-tuning or compute-intensive procedures. Despite the red-teaming efforts and inherent potential risks associated with the LVLMs, exploring vulnerabilities of LVLMs remains nascent and yet to be fully addressed in a systematic manner. This study systematically examines the vulnerabilities of open-source LVLMs, including LLaVA, InstructBLIP, Fuyu, and Qwen, using adversarial prompt strategies that simulate real-world social manipulation tactics informed by social theories. Our findings show that (i) toxicity and insulting are the most prevalent behaviors, with the mean rates of 16.13% and 9.75%, respectively; (ii) Qwen-VL-Chat, LLaVA-v1.6-Vicuna-7b, and InstructBLIP-Vicuna-7b are the most vulnerable models, exhibiting toxic response rates of 21.50%, 18.30% and 17.90%, and insulting responses of 13.40%, 11.70% and 10.10%, respectively; (iii) prompting strategies incorporating dark humor and multimodal toxic prompt completion significantly elevated these vulnerabilities. Despite being fine-tuned for safety, these models still generate content with varying degrees of toxicity when prompted with adversarial inputs, highlighting the urgent need for enhanced safety mechanisms and robust guardrails in LVLM development.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 51 of its 121 references that are in the map. First posted on arXiv 14 Jan 2025.

Builds on

51 of 121 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends10 Jul
  2. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models14 Mar
  3. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models5 Feb
  4. 2024Red Teaming Visual Language Models23 Jan
  5. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep
  6. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
  7. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  7. 2023Qwen Technical Reportcited by 3,344
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  10. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  11. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  12. 2022Red Teaming Language Models with Language Modelscited by 1,026
32 more
  1. 2023A Survey on Multimodal Large Language Modelscited by 909
  2. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  3. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  4. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  5. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  6. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 490
  7. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 429
  8. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  9. 2023Are aligned neural networks adversarially aligned?cited by 315
  10. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  11. 2023Challenges and Applications of Large Language Modelscited by 254
  12. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 214
  13. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  14. 2023Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challengescited by 131
  15. 2023Evaluation and Analysis of Hallucination in Large Vision-Language Modelscited by 99
  16. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  17. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 89
  18. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Modelscited by 76
  19. 2022Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examplescited by 69
  20. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directionscited by 68
  21. 2024Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Imagescited by 55
  22. 2024RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agentcited by 44
  23. 2023Visual Instruction Tuning with Polite Flamingocited by 38
  24. 2021Defining and Detecting Toxicity on Social Media: Context and Knowledge are Keycited by 29
  25. 2022What do you MEME? Generating Explanations for Visual Semantic Role Labelling in Memescited by 18
  26. 2020ALONE: A Dataset for Toxic Behavior among Adolescents on Twittercited by 17
  27. 2024LLMGuard: Guarding Against Unsafe LLM Behaviorcited by 13
  28. 2024Uncovering Bias in Large Vision-Language Models with Counterfactualscited by 9
  29. 2024Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memescited by 8
  30. 2022Measuring the Prevalence of Anti-Social Behavior in Online Communitiescited by 6
  31. 2019Modeling Islamist Extremist Communications on Social Media using Contextual Dimensions: Religion, Ideology, and Hatecited by 4
  32. 2024The Adaptive Strategies of Anti-Kremlin Digital Dissent in Telegram during the Russian Invasion of Ukrainecited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Large Language Models for Cancer Communication: Evaluating Linguistic Quality, Safety, and Accessibility in Generative AIcited by 1
  2. 2025From Reddit to Generative AI: Evaluating Large Language Models for Anxiety Support Fine-tuned on Social Media Datacited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attack
  2. 2025ELITE: Enhanced Language-Image Toxicity Evaluation for Safety
  3. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations
  4. 2025Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Study
  5. 2025Spot Risks Before Speaking! Unraveling Safety Attention Heads in Large Vision-Language Models
  6. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Survey
  7. 2026STARE: Step-wise Temporal Alignment and Red-teaming Engine for Multi-modal Toxicity Attack
  8. 2024The Dark Side of Human Feedback: Poisoning Large Language Models via User Inputs