mlsanityData to 7 Oct 2026

Visual Adversarial Examples Jailbreak Aligned Large Language Models

arXiv 2306.13213 ↗First posted 22 Jun 2023cs.CR cs.CL cs.LGcited by 307

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 2 more
Abstract (arXiv)

Recently, there has been a surge of interest in integrating vision into Large Language Models (LLMs), exemplified by Visual Language Models (VLMs) such as Flamingo and GPT-4. This paper sheds light on the security and safety implications of this trend. First, we underscore that the continuous and high-dimensional nature of the visual input makes it a weak link against adversarial attacks, representing an expanded attack surface of vision-integrated LLMs. Second, we highlight that the versatility of LLMs also presents visual attackers with a wider array of achievable adversarial objectives, extending the implications of security failures beyond mere misclassification. As an illustration, we present a case study in which we exploit visual adversarial examples to circumvent the safety guardrail of aligned LLMs with integrated vision. Intriguingly, we discover that a single visual adversarial example can universally jailbreak an aligned LLM, compelling it to heed a wide range of harmful instructions that it otherwise would not) and generate harmful content that transcends the narrow scope of a `few-shot' derogatory corpus initially employed to optimize the adversarial example. Our study underscores the escalating adversarial risks associated with the pursuit of multimodality. Our findings also connect the long-studied adversarial vulnerabilities of neural networks to the nascent field of AI alignment. The presented attack suggests a fundamental adversarial challenge for AI alignment, especially in light of the emerging trend toward multimodality in frontier foundation models.

Key facts. Cited by 307 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 60 of its 89 references that are in the map. First posted on arXiv 22 Jun 2023.

Builds on

60 of 89 references.

References · 60

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  11. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  12. 2013Intriguing properties of neural networkscited by 5,114
38 more
  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  10. 2015VQA: Visual Question Answeringcited by 2,489
  11. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  12. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  13. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  14. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  15. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  16. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  17. 2022Red Teaming Language Models with Language Modelscited by 983
  18. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  19. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  20. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  21. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  22. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  23. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  24. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  25. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499
  26. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  27. 2022Diffusion Models for Adversarial Purificationcited by 469
  28. 2018Generating Natural Language Adversarial Examplescited by 422
  29. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  30. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  31. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  32. 2023Large Language Models as Tool Makerscited by 293
  33. 2018Audio Adversarial Examples: Targeted Attacks on Speech-to-Textcited by 268
  34. 2022Predictability and Surprise in Large Generative Modelscited by 209
  35. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  36. 2017Generating Natural Adversarial Examplescited by 182
  37. 2021Challenges in Detoxifying Language Modelscited by 169
  38. 2021Alignment of Language Agentscited by 167

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 73; 2025: 116; 2026: 103.

307 papers.

Sort:

By area:

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  4. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  5. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  6. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  7. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  8. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 260
  9. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
  10. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  11. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  12. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
38 more
  1. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  2. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
  3. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113
  4. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 103
  5. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 103
  6. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Modelscited by 102
  7. 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attackscited by 89
  8. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 89
  9. 2024Coercing LLMs to do and reveal (almost) anythingcited by 88
  10. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 82
  11. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  12. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  13. 2024Embodied LLM Agents Learn to Cooperate in Organized Teamscited by 81
  14. 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
  15. 2024Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Roboticscited by 68
  16. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  17. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  18. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 57
  19. 2024LLMmap: Fingerprinting For Large Language Modelscited by 56
  20. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 51
  21. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 48
  22. 2023Can LLMs Follow Simple Rules?cited by 45
  23. 2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Modelscited by 45
  24. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  25. 2024AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Modelscited by 43
  26. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42
  27. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistencycited by 42
  28. 2023Misusing Tools in Large Language Models With Visual Adversarial Examplescited by 40
  29. 2024Visual Adversarial Attack on Vision-Language Models for Autonomous Drivingcited by 35
  30. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselvescited by 34
  31. 2025Backdoor Cleaning without External Guidance in MLLM Fine-tuningcited by 34
  32. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  33. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 33
  34. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  35. 2025Distraction is All You Need for Multimodal Large Language Model Jailbreakingcited by 31
  36. 2025Adversarial Attacks on Robotic Vision Language Action Modelscited by 31
  37. 2025Safety Guardrails for LLM-Enabled Robotscited by 30
  38. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Modelscited by 29

50 most cited of 307 All 307 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Models
  2. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models
  3. 2026Seeing No Evil: Blinding Large Vision-Language Models to Safety Instructions via Adversarial Attention Hijacking
  4. 2025Scaling Patterns in Adversarial Alignment: Evidence from Multi-LLM Jailbreak Experiments
  5. 2025Universal Adversarial Attack on Aligned Multimodal LLMs
  6. 2024White-box Multimodal Jailbreaks Against Large Vision-Language Models
  7. 2024Learning To See But Forgetting To Follow: Visual Instruction Tuning Makes LLMs More Prone To Jailbreak Attacks
  8. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks