mlsanityData to 7 Oct 2026

How Robust is Google's Bard to Adversarial Image Attacks?

arXiv 2309.11751 ↗First posted 21 Sep 2023cs.CV cs.AI cs.CRcited by 177

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) that integrate text and other modalities (especially vision) have achieved unprecedented performance in various multimodal tasks. However, due to the unsolved adversarial robustness problem of vision models, MLLMs can have more severe safety and security risks by introducing the vision inputs. In this work, we study the adversarial robustness of Google's Bard, a competitive chatbot to ChatGPT that released its multimodal capability recently, to better understand the vulnerabilities of commercial MLLMs. By attacking white-box surrogate vision encoders or MLLMs, the generated adversarial examples can mislead Bard to output wrong image descriptions with a 22% success rate based solely on the transferability. We show that the adversarial examples can also attack other MLLMs, e.g., a 26% attack success rate against Bing Chat and a 86% attack success rate against ERNIE bot. Moreover, we identify two defense mechanisms of Bard, including face detection and toxicity detection of images. We design corresponding attacks to evade these defenses, demonstrating that the current defenses of Bard are also vulnerable. We hope this work can deepen our understanding on the robustness of MLLMs and facilitate future research on defenses. Our code is available at https://github.com/thu-ml/Attack-Bard. Update: GPT-4V is available at October 2023. We further evaluate its robustness under the same set of adversarial examples, achieving a 45% attack success rate.

Key facts. Cited by 177 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 55 of its 64 references that are in the map. First posted on arXiv 21 Sep 2023.

Builds on

55 of 64 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  11. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  12. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
38 more
  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  5. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  10. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  11. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  12. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  13. 2023PaLM 2 Technical Reportcited by 1,154
  14. 2022Red Teaming Language Models with Language Modelscited by 983
  15. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  16. 2023A Survey on Multimodal Large Language Modelscited by 898
  17. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  18. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  19. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  20. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 591
  21. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
  22. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  23. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499
  24. 2022Diffusion Models for Adversarial Purificationcited by 469
  25. 2019Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attackscited by 440
  26. 2019Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attackscited by 352
  27. 2023PandaGPT: One Model To Instruction-Follow Them Allcited by 330
  28. 2023Are aligned neural networks adversarially aligned?cited by 305
  29. 2021Enhancing the Transferability of Adversarial Attacks through Variance Tuningcited by 242
  30. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  31. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  32. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  33. 2017S3FD: Single Shot Scale-invariant Face Detectorcited by 142
  34. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
  35. 2022Frequency Domain Model Augmentation for Adversarial Attackcited by 128
  36. 2018DSFD: Dual Shot Face Detectorcited by 115
  37. 2023Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Modelscited by 100
  38. 2023Rethinking Model Ensemble in Transfer-based Adversarial Attackscited by 88

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 60; 2025: 63; 2026: 47.

177 papers.

Sort:

By area:

  1. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 208
  2. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
  3. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  4. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Modelscited by 121
  5. 2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Modelscited by 119
  6. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 118
  7. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  8. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113
  9. 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
  10. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 103
  11. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 103
  12. 2024NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samplescited by 94
38 more
  1. 2023Jatmo: Prompt Injection Defense by Task-Specific Finetuningcited by 89
  2. 2024WAVES: Benchmarking the Robustness of Image Watermarkscited by 89
  3. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  4. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
  5. 2024Best-of-N Jailbreakingcited by 71
  6. 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
  7. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  8. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  9. 2023Jailbreaking GPT-4V via Self-Adversarial Attacks with System Promptscited by 62
  10. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
  11. 2025Trading Inference-Time Compute for Adversarial Robustnesscited by 60
  12. 2024Jailbreak Large Vision-Language Models Through Multi-Modal Linkagecited by 53
  13. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Modelscited by 48
  14. 2023A Survey on Transferability of Adversarial Examples across Deep Neural Networkscited by 43
  15. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42
  16. 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 42
  17. 2024Few-Shot Adversarial Prompt Learning on Vision-Language Modelscited by 39
  18. 2024Boosting Jailbreak Attack with Momentumcited by 38
  19. 2024Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attackscited by 37
  20. 2024Efficient Generation of Targeted and Transferable Adversarial Examples for Vision-Language Models Via Diffusion Modelscited by 36
  21. 2024Test-Time Backdoor Attacks on Multimodal Large Language Modelscited by 35
  22. 2024Visual Adversarial Attack on Vision-Language Models for Autonomous Drivingcited by 35
  23. 2025A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o1cited by 34
  24. 2025Backdoor Cleaning without External Guidance in MLLM Fine-tuningcited by 34
  25. 2025Safety Guardrails for LLM-Enabled Robotscited by 30
  26. 2025HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden Statescited by 29
  27. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agentscited by 28
  28. 2024Failures to Find Transferable Image Jailbreaks Between Vision-Language Modelscited by 27
  29. 2024B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructionscited by 26
  30. 2024On the Duality Between Sharpness-Aware Minimization and Adversarial Trainingcited by 24
  31. 2024Adversarial Robustness for Visual Grounding of Multimodal Large Language Modelscited by 23
  32. 2024Benchmarking Large Multimodal Models against Common Corruptionscited by 22
  33. 2024Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Modelcited by 22
  34. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  35. 2023InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Modelscited by 21
  36. 2024MirrorCheck: Efficient Adversarial Defense for Vision-Language Modelscited by 19
  37. 2024RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Promptingcited by 19
  38. 2025ADVEDM:Fine-grained Adversarial Attack against VLM-based Embodied Agentscited by 19

50 most cited of 177 All 177 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025
  2. 2024White-box Multimodal Jailbreaks Against Large Vision-Language Models
  3. 2026Breaking the weakest link to evade vision language models
  4. 2024Multimodal Adversarial Defense for Vision-Language Models by Leveraging One-To-Many Relationships
  5. 2024Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective
  6. 2026Investigating Adversarial Robustness of Multi-modal Large Language Models
  7. 2025Universal Adversarial Attack on Aligned Multimodal LLMs
  8. 2026Dynamic Adversarial Reinforcement Learning for Robust Multimodal Large Language Models