mlsanityData to 7 Oct 2026

On the Adversarial Robustness of Multi-Modal Foundation Models

arXiv 2308.10741 ↗First posted 21 Aug 2023cs.LG cs.AI cs.CRcited by 142

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

Multi-modal foundation models combining vision and language models such as Flamingo or GPT-4 have recently gained enormous interest. Alignment of foundation models is used to prevent models from providing toxic or harmful output. While malicious users have successfully tried to jailbreak foundation models, an equally important question is if honest users could be harmed by malicious third-party content. In this paper we show that imperceivable attacks on images in order to change the caption output of a multi-modal foundation model can be used by malicious content providers to harm honest users e.g. by guiding them to malicious websites or broadcast fake information. This indicates that countermeasures to adversarial attacks should be used by any deployed multi-modal foundation model.

Key facts. Cited by 142 papers in the map (data through 7 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 26 of its 42 references that are in the map. First posted on arXiv 21 Aug 2023.

Builds on

26 of 42 references.

References · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  8. 2013Intriguing properties of neural networkscited by 5,114
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  11. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  12. 2015VQA: Visual Question Answeringcited by 2,489
14 more
  1. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  2. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  3. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  4. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  5. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  6. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  7. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  8. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  9. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  10. 2023Are aligned neural networks adversarially aligned?cited by 305
  11. 2022mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connectionscited by 185
  12. 2023Grounding Language Models to Images for Multimodal Inputs and Outputscited by 129
  13. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 117
  14. 2020Adversarial Evaluation of Multimodal Models under Realistic Gray Box Assumptioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 48; 2025: 52; 2026: 35.

142 papers.

Sort:

By area:

  1. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  2. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  3. 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 177
  4. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  5. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  6. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Modelscited by 121
  7. 2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Modelscited by 119
  8. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 118
  9. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  10. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 103
  11. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  12. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
38 more
  1. 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
  2. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  3. 2023Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competitioncited by 65
  4. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
  5. 2024White-box Multimodal Jailbreaks Against Large Vision-Language Modelscited by 57
  6. 2024One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Modelscited by 48
  7. 2024GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Modelscited by 44
  8. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  9. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42
  10. 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 42
  11. 2024Core Knowledge Deficits in Multi-Modal Language Modelscited by 38
  12. 2024Test-Time Backdoor Attacks on Multimodal Large Language Modelscited by 35
  13. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  14. 2025Backdoor Cleaning without External Guidance in MLLM Fine-tuningcited by 34
  15. 2024R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?cited by 29
  16. 2024Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Modelscited by 27
  17. 2024B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructionscited by 26
  18. 2025MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attackscited by 24
  19. 2024Cross-Modal Safety Alignment: Is textual unlearning all you need?cited by 22
  20. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  21. 2025How Do Vision-Language Models Process Conflicting Information Across Modalities?cited by 22
  22. 2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
  23. 2024The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defensecited by 20
  24. 2024MirrorCheck: Efficient Adversarial Defense for Vision-Language Modelscited by 19
  25. 2024Order Matters: Exploring Order Sensitivity in Multimodal Large Language Modelscited by 19
  26. 2025ADVEDM:Fine-grained Adversarial Attack against VLM-based Embodied Agentscited by 19
  27. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Securitycited by 18
  28. 2024Quantifying and Enhancing Multi-modal Robustness with Modality Preferencecited by 16
  29. 2024HiddenGuard: Fine-Grained Safe Generation with Specialized Representation Routercited by 16
  30. 2024A Survey on Safe Multi-Modal Learning Systemcited by 15
  31. 2024BAPLe: Backdoor Attacks on Medical Foundational Models using Prompt Learningcited by 15
  32. 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 14
  33. 2025Survey of Adversarial Robustness in Multimodal Large Language Modelscited by 14
  34. 2024Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial Attackscited by 13
  35. 2024Sim-CLIP: Unsupervised Siamese Adversarial Fine-Tuning for Robust and Semantically-Rich Vision-Language Modelscited by 12
  36. 2025Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generationcited by 12
  37. 2024Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Modelscited by 11
  38. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 11

50 most cited of 142 All 142 (JSON)