mlsanityData to 7 Oct 2026

On Evaluating Adversarial Robustness of Large Vision-Language Models

arXiv 2305.16934 ↗First posted 26 May 2023cs.CV cs.CL cs.CRcited by 281

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Large vision-language models (VLMs) such as GPT-4 have achieved unprecedented performance in response generation, especially with visual inputs, enabling more creative and adaptable interaction than large language models such as ChatGPT. Nonetheless, multimodal generation exacerbates safety concerns, since adversaries may successfully evade the entire system by subtly manipulating the most vulnerable modality (e.g., vision). To this end, we propose evaluating the robustness of open-source large VLMs in the most realistic and high-risk setting, where adversaries have only black-box system access and seek to deceive the model into returning the targeted responses. In particular, we first craft targeted adversarial examples against pretrained models such as CLIP and BLIP, and then transfer these adversarial examples to other VLMs such as MiniGPT-4, LLaVA, UniDiffuser, BLIP-2, and Img2Prompt. In addition, we observe that black-box queries on these VLMs can further improve the effectiveness of targeted evasion, resulting in a surprisingly high success rate for generating targeted responses. Our findings provide a quantitative understanding regarding the adversarial vulnerability of large VLMs and call for a more thorough examination of their potential security flaws before deployment in practice. Code is at https://github.com/yunqing-me/AttackVLM.

Key facts. Cited by 281 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 90 of its 108 references that are in the map. First posted on arXiv 26 May 2023.

Builds on

90 of 108 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Towards Adversarial Attack on Vision-Language Pre-training Models19 Jun

Other references · 89

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2022Training language models to follow instructions with human feedbackcited by 15,712
  9. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  10. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  11. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  12. 2023Visual Instruction Tuningcited by 7,792
38 more
  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  13. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  14. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  15. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  16. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  17. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  18. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  19. 2022Red Teaming Language Models with Language Modelscited by 983
  20. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  21. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
  22. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  23. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  24. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  25. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  26. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  27. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  28. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  29. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  30. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
  31. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 591
  32. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  33. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  34. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
  35. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  36. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  37. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499
  38. 2019On Evaluating Adversarial Robustnesscited by 442

50 most cited of 89 All 89 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 27; 2024: 82; 2025: 102; 2026: 70.

281 papers.

Sort:

By area:

  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  3. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  4. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  5. 2023A Survey on Multimodal Large Language Modelscited by 898
  6. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  7. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 260
  8. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  9. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  10. 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 177
  11. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  12. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
38 more
  1. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  2. 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 125
  3. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Modelscited by 121
  4. 2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Modelscited by 119
  5. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 118
  6. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  7. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 103
  8. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 103
  9. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?cited by 86
  10. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 82
  11. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  12. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
  13. 2024BRAVE: Broadening the visual encoding of vision-language modelscited by 68
  14. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  15. 2024Introducing v0.5 of the AI Safety Benchmark from MLCommonscited by 60
  16. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  17. 2024Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustnesscited by 48
  18. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Modelscited by 48
  19. 2024One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Modelscited by 48
  20. 2024GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Modelscited by 44
  21. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  22. 2023A Survey on Transferability of Adversarial Examples across Deep Neural Networkscited by 43
  23. 2024AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Modelscited by 43
  24. 2023Adversarial Prompt Tuning for Vision-Language Modelscited by 42
  25. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42
  26. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistencycited by 42
  27. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 42
  28. 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 42
  29. 2024An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Modelscited by 40
  30. 2024Core Knowledge Deficits in Multi-Modal Language Modelscited by 38
  31. 2024Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attackscited by 37
  32. 2024Efficient Generation of Targeted and Transferable Adversarial Examples for Vision-Language Models Via Diffusion Modelscited by 36
  33. 2024Test-Time Backdoor Attacks on Multimodal Large Language Modelscited by 35
  34. 2024Visual Adversarial Attack on Vision-Language Models for Autonomous Drivingcited by 35
  35. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 34
  36. 2025A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o1cited by 34
  37. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  38. 2025Distraction is All You Need for Multimodal Large Language Model Jailbreakingcited by 31

50 most cited of 281 All 281 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Investigating Adversarial Robustness of Multi-modal Large Language Models
  2. 2024A Hybrid Defense Strategy for Boosting Adversarial Robustness in Vision-Language Models
  3. 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attacks
  4. 2026What Makes VLMs Robust? Towards Reconciling Robustness and Accuracy in Vision-Language Models
  5. 2025Double Visual Defense: Adversarial Pre-training and Instruction Tuning for Improving Vision-Language Model Robustness
  6. 2025Navigating the Trade-off: A Synthesis of Defensive Strategies for Zero-Shot Adversarial Robustness in Vision-Language Models
  7. 2026Crafting Adversarial Inputs for Large Vision-Language Models Using Black-Box Optimization
  8. 2025A Framework for Evaluating Vision-Language Model Safety: Building Trust in AI for Public Sector Applications