Adversarial Diffusion Across Modalities: A Fusion Survey of Attacks, Defenses, and Evaluation for Text, Vision, and Vision-Language Models
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
also in 1 more
Abstract (arXiv)
Adversarial evaluation of AI systems has matured along four largely disconnected tracks: diffusion-based attacks on text and large language models (LLMs), diffusion-based attacks on image classifiers, jailbreak pipelines against vision-language models, and diffusion-based input purification defenses. Each has developed its own vocabulary, threat models, and benchmarks, with denoising diffusion models emerging as a shared generative mechanism whose recipes are now actively ported between communities. This survey performs an information-fusion exercise at the meta-research level: we integrate these four tracks into a single conceptual framework with a unified taxonomy, evaluation criteria, and research agenda, focusing on the LLM-side slice. We catalog fifty published papers across four scope areas (text/LLM, image classifier, vision-language model, defense), plus four diffusion-LLM-as-victim entries and ten non-diffusion baselines against which any new attack must be compared. We propose a six-class taxonomy of diffusion roles in adversarial pipelines, augmented by a threat-model axis recording attacker knowledge, query budget, and target accessibility, and apply a five-dimension framework (attack success rate, transferability, query budget, perplexity, defense-evasion) uniformly across modalities. The review adopts a dual attacker-defender perspective: alongside the attack catalog we cover four diffusion-based defenses that form the natural evaluation backdrop for new attacks. Our critical analysis identifies five recurring weaknesses of the current LLM-side literature, and we close with a research agenda of open questions and concrete experimental designs. The companion catalog and spreadsheet are released with the paper. We are explicit that this is a narrative review with quality assessment, not a PRISMA-compliant systematic review, and discuss the implications for replication.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 49 of its 68 references that are in the map. First posted on arXiv 25 Jun 2026.
Builds on
49 of 68 references.
Closest ancestors · 7
Same direction, newest first
- 2025CoDefend: Cross-Modal Collaborative Defense via Diffusion Purification and Prompt Optimization13 Oct
- 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves29 Oct
- 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt6 Jun
- 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
- 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
- 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
- 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May
Other references · 42
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
30 more
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2022Red Teaming Language Models with Language Modelscited by 1,026
- 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
- 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
- 2022Diffusion Models for Adversarial Purificationcited by 476
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
- 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
- 2023Diffusion Models for Imperceptible and Transferable Adversarial Attackcited by 94
- 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
- 2023FLIRT: Feedback Loop In-context Red Teamingcited by 83
- 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 68
- 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
- 2024Against The Achilles' Heel: A Survey on Red Teaming for Generative Modelscited by 59
- 2023DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purificationcited by 36
- 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMscited by 34
- 2025Jailbreaking Large Language Diffusion Models: Revealing Hidden Safety Flaws in Diffusion-Based Text Generationcited by 26
- 2023Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Modelcited by 21
- 2024Gradient-based Jailbreak Images for Multimodal Fusion Modelscited by 14
- 2024AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion modelscited by 10
- 2025Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerabilitycited by 10
- 2025Diffusion LLMs are Natural Adversaries for any LLMcited by 7
- 2025NatADiff: Adversarial Boundary Guidance for Natural Adversarial Diffusioncited by 6
- 2024AdvLogo: Adversarial Patch Attack against Object Detectors based on Diffusion Modelscited by 5
- 2025Text-Diffusion Red-Teaming of Large Language Models: Unveiling Harmful Behaviors with Proximity Constraintscited by 5
- 2024BadPatch: Diffusion-Based Generation of Physical Adversarial Patchescited by 4
- 2025SemDiff: Generating Natural Unrestricted Adversarial Examples via Semantic Attributes Optimization in Diffusion Modelscited by 3
- 2025Enhancing Diffusion-based Unrestricted Adversarial Attacks via Adversary Preferences Alignmentcited by 3
- 2025ScoreAdv: Score-based Targeted Generation of Natural Adversarial Examples via Diffusion Modelscited by 3
- 2024MaskPure: Improving Defense Against Text Adversaries with Stochastic Purificationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Survey
- 2026PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Models
- 2025Diffusion-based Cumulative Adversarial Purification for Vision Language Models
- 2025Diffusion or Non-Diffusion Adversarial Defenses: Rethinking the Relation between Classifier and Adversarial Purifier
- 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Survey
- 2024Adversarial Examples are Misaligned in Diffusion Model Manifolds
- 2023Semantic Adversarial Attacks via Diffusion Models
- 2024Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey