mlsanityData to 7 Oct 2026

Breaking Free: How to Hack Safety Guardrails in Black-Box Diffusion Models!

arXiv 2402.04699 ↗First posted 7 Feb 2024cs.CV cs.AI cs.LGcited by 0

On the map: Diffusion models → Robustness, privacy and adversarial aspects

also in 1 more
Abstract (arXiv)

Deep neural networks can be exploited using natural adversarial samples, which do not impact human perception. Current approaches often rely on deep neural networks' white-box nature to generate these adversarial samples or synthetically alter the distribution of adversarial samples compared to the training distribution. In contrast, we propose EvoSeed, a novel evolutionary strategy-based algorithmic framework for generating photo-realistic natural adversarial samples. Our EvoSeed framework uses auxiliary Conditional Diffusion and Classifier models to operate in a black-box setting. We employ CMA-ES to optimize the search for an initial seed vector, which, when processed by the Conditional Diffusion Model, results in the natural adversarial sample misclassified by the Classifier Model. Experiments show that generated adversarial images are of high image quality, raising concerns about generating harmful content bypassing safety classifiers. Our research opens new avenues to understanding the limitations of current safety mechanisms and the risk of plausible attacks against classifier systems using image generation. Project Website can be accessed at: https://shashankkotyan.github.io/EvoSeed.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 42 of its 52 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

42 of 52 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023SD-NAE: Generating Natural Adversarial Examples with Stable Diffusion21 Nov
  2. 2023Diffusion Models for Imperceptible and Transferable Adversarial Attack14 May
  3. 2023Better Diffusion Models Further Improve Adversarial Training9 Feb
  4. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models9 Nov

Other references · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2022Generative Adversarial Networkscited by 5,603
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
26 more
  1. 2014Conditional Generative Adversarial Netscited by 2,152
  2. 2019Natural Adversarial Examplescited by 1,401
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  4. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  5. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  6. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  7. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  8. 2023Adversarial Diffusion Distillationcited by 741
  9. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  10. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  11. 2017One pixel attack for fooling deep neural networkscited by 579
  12. 2018Generating Adversarial Examples with Adversarial Networkscited by 256
  13. 2018Spatially Transformed Adversarial Examplescited by 195
  14. 2017Generating Natural Adversarial Examplescited by 182
  15. 2017EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examplescited by 154
  16. 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 150
  17. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  18. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesiscited by 136
  19. 2018Constructing Unrestricted Adversarial Examples with Generative Modelscited by 135
  20. 2022Revisiting Weakly Supervised Pre-Training of Visual Perception Modelscited by 116
  21. 2019ColorFool: Semantic Adversarial Colorizationcited by 89
  22. 2021Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidancecited by 84
  23. 2021A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustnesscited by 62
  24. 2023Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Searchcited by 25
  25. 2019AdvGAN++ : Harnessing latent layers for adversary generationcited by 20
  26. 2023Synthetic Shifts to Initial Seed Vector Exposes the Brittle Nature of Latent-Based Diffusion Modelscited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Explore the vulnerability of black-box models via diffusion models
  2. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now
  3. 2024Deceptive Diffusion: Generating Synthetic Adversarial Examples
  4. 2022Threat Model-Agnostic Adversarial Defense using Diffusion Models
  5. 2025Embedding Hidden Adversarial Capabilities in Pre-Trained Diffusion Models
  6. 2023Diffusion-Based Adversarial Sample Generation for Improved Stealthiness and Controllability
  7. 2025Diffusion or Non-Diffusion Adversarial Defenses: Rethinking the Relation between Classifier and Adversarial Purifier
  8. 2025Lightweight Safety Guardrails via Synthetic Data and RL-guided Adversarial Training