mlsanityData to 7 Oct 2026

Red-Teaming the Stable Diffusion Safety Filter

arXiv 2210.04610 ↗First posted 3 Oct 2022cs.AI cs.CR cs.CVcited by 257

On the map: Diffusion models → Robustness, privacy and adversarial aspects

also in 1 more
Abstract (arXiv)

Stable Diffusion is a recent open-source image generation model comparable to proprietary models such as DALLE, Imagen, or Parti. Stable Diffusion comes with a safety filter that aims to prevent generating explicit images. Unfortunately, the filter is obfuscated and poorly documented. This makes it hard for users to prevent misuse in their applications, and to understand the filter's limitations and improve it. We first show that it is easy to generate disturbing content that bypasses the safety filter. We then reverse-engineer the filter and find that while it aims to prevent sexual content, it ignores violence, gore, and other similarly disturbing content. Based on our analysis, we argue safety measures in future model releases should strive to be fully open and properly documented to stimulate security contributions from the community.

Key facts. Cited by 257 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 5 of its 20 references that are in the map. First posted on arXiv 3 Oct 2022.

Builds on

5 of 20 references.

References · 5

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2018Model Cards for Model Reportingcited by 1,068

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 37; 2024: 88; 2025: 75; 2026: 55.

257 papers.

Sort:

By area:

  1. 2023Erasing Concepts from Diffusion Modelscited by 530
  2. 2023Unified Concept Editing in Diffusion Modelscited by 377
  3. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  4. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 298
  5. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  6. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  7. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 260
  8. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?cited by 239
  9. 2023Understanding and Mitigating Copying in Diffusion Modelscited by 205
  10. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 204
  11. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 184
  12. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 175
38 more
  1. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  2. 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationscited by 164
  3. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 164
  4. 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Promptscited by 161
  5. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  6. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  7. 2024Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Modelscited by 123
  8. 2024EraseAnything: Enabling Concept Erasure in Rectified Flow Transformerscited by 93
  9. 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 81
  10. 2023FLIRT: Feedback Loop In-context Red Teamingcited by 81
  11. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  12. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 75
  13. 2023Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generationcited by 69
  14. 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Modelscited by 68
  15. 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
  16. 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Modelscited by 64
  17. 2023The Gradient of Generative AI Release: Methods and Considerationscited by 63
  18. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  19. 2024GuardT2I: Defending Text-to-Image Models from Adversarial Promptscited by 61
  20. 2022Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Modelscited by 56
  21. 2024R.A.C.E.: Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Modelcited by 56
  22. 2024Erasing Undesirable Influence in Diffusion Modelscited by 52
  23. 2024MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Controlcited by 52
  24. 2024UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Imagescited by 52
  25. 2024Scissorhands: Scrub Data Influence via Connection Sensitivity in Networkscited by 50
  26. 2024Direct Unlearning Optimization for Robust and Safe Text-to-Image Modelscited by 50
  27. 2024Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservationcited by 48
  28. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 47
  29. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 46
  30. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancementcited by 43
  31. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistencycited by 42
  32. 2024STEREO: A Two-Stage Framework for Adversarially Robust Concept Erasing from Text-to-Image Diffusion Modelscited by 39
  33. 2024Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Conceptscited by 39
  34. 2024Precise, Fast, and Low-cost Concept Erasure in Value Space: Orthogonal Complement Matterscited by 38
  35. 2024Perception-guided Jailbreak against Text-to-Image Modelscited by 37
  36. 2024Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attackscited by 37
  37. 2023Mass-Producing Failures of Multimodal Systems with Language Modelscited by 36
  38. 2024Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Surveycited by 34

50 most cited of 257 All 257 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Anti-Diffusion: Preventing Abuse of Modifications of Diffusion-Based Models
  2. 2024Safe-SD: Safe and Traceable Stable Diffusion with Text Prompt Trigger for Invisible Generative Watermarking
  3. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models
  4. 2023Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation
  5. 2026Purify Once, Edit Freely: Breaking Image Protections under Model Mismatch
  6. 2023Can Protective Perturbation Safeguard Personal Data from Being Exploited by Stable Diffusion?
  7. 2023CRoSS: Diffusion Model Makes Controllable, Robust and Secure Image Steganography
  8. 2023Towards More Realistic Membership Inference Attacks on Large Diffusion Models