mlsanityData to 14 Oct 2026

Adversarial Diffusion Across Modalities: A Fusion Survey of Attacks, Defenses, and Evaluation for Text, Vision, and Vision-Language Models

arXiv 2606.26566 ↗First posted 25 Jun 2026cs.CR cs.CLcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Adversarial evaluation of AI systems has matured along four largely disconnected tracks: diffusion-based attacks on text and large language models (LLMs), diffusion-based attacks on image classifiers, jailbreak pipelines against vision-language models, and diffusion-based input purification defenses. Each has developed its own vocabulary, threat models, and benchmarks, with denoising diffusion models emerging as a shared generative mechanism whose recipes are now actively ported between communities. This survey performs an information-fusion exercise at the meta-research level: we integrate these four tracks into a single conceptual framework with a unified taxonomy, evaluation criteria, and research agenda, focusing on the LLM-side slice. We catalog fifty published papers across four scope areas (text/LLM, image classifier, vision-language model, defense), plus four diffusion-LLM-as-victim entries and ten non-diffusion baselines against which any new attack must be compared. We propose a six-class taxonomy of diffusion roles in adversarial pipelines, augmented by a threat-model axis recording attacker knowledge, query budget, and target accessibility, and apply a five-dimension framework (attack success rate, transferability, query budget, perplexity, defense-evasion) uniformly across modalities. The review adopts a dual attacker-defender perspective: alongside the attack catalog we cover four diffusion-based defenses that form the natural evaluation backdrop for new attacks. Our critical analysis identifies five recurring weaknesses of the current LLM-side literature, and we close with a research agenda of open questions and concrete experimental designs. The companion catalog and spreadsheet are released with the paper. We are explicit that this is a narrative review with quality assessment, not a PRISMA-compliant systematic review, and discuss the implications for replication.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 49 of its 68 references that are in the map. First posted on arXiv 25 Jun 2026.

Builds on

49 of 68 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025CoDefend: Cross-Modal Collaborative Defense via Diffusion Purification and Prompt Optimization13 Oct
  2. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves29 Oct
  3. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt6 Jun
  4. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  5. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  6. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
  7. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 42

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  11. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  12. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
30 more
  1. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  2. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  3. 2022Red Teaming Language Models with Language Modelscited by 1,026
  4. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  5. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  6. 2022Diffusion Models for Adversarial Purificationcited by 476
  7. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
  8. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  9. 2023Diffusion Models for Imperceptible and Transferable Adversarial Attackcited by 94
  10. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
  11. 2023FLIRT: Feedback Loop In-context Red Teamingcited by 83
  12. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 68
  13. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  14. 2024Against The Achilles' Heel: A Survey on Red Teaming for Generative Modelscited by 59
  15. 2023DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purificationcited by 36
  16. 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMscited by 34
  17. 2025Jailbreaking Large Language Diffusion Models: Revealing Hidden Safety Flaws in Diffusion-Based Text Generationcited by 26
  18. 2023Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Modelcited by 21
  19. 2024Gradient-based Jailbreak Images for Multimodal Fusion Modelscited by 14
  20. 2024AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion modelscited by 10
  21. 2025Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerabilitycited by 10
  22. 2025Diffusion LLMs are Natural Adversaries for any LLMcited by 7
  23. 2025NatADiff: Adversarial Boundary Guidance for Natural Adversarial Diffusioncited by 6
  24. 2024AdvLogo: Adversarial Patch Attack against Object Detectors based on Diffusion Modelscited by 5
  25. 2025Text-Diffusion Red-Teaming of Large Language Models: Unveiling Harmful Behaviors with Proximity Constraintscited by 5
  26. 2024BadPatch: Diffusion-Based Generation of Physical Adversarial Patchescited by 4
  27. 2025SemDiff: Generating Natural Unrestricted Adversarial Examples via Semantic Attributes Optimization in Diffusion Modelscited by 3
  28. 2025Enhancing Diffusion-based Unrestricted Adversarial Attacks via Adversary Preferences Alignmentcited by 3
  29. 2025ScoreAdv: Score-based Targeted Generation of Natural Adversarial Examples via Diffusion Modelscited by 3
  30. 2024MaskPure: Improving Defense Against Text Adversaries with Stochastic Purificationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2024Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Survey
  2. 2026PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Models
  3. 2025Diffusion-based Cumulative Adversarial Purification for Vision Language Models
  4. 2025Diffusion or Non-Diffusion Adversarial Defenses: Rethinking the Relation between Classifier and Adversarial Purifier
  5. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Survey
  6. 2024Adversarial Examples are Misaligned in Diffusion Model Manifolds
  7. 2023Semantic Adversarial Attacks via Diffusion Models
  8. 2024Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey