mlsanityData to 14 Oct 2026

Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservation

arXiv 2410.15618 ↗First posted 21 Oct 2024cs.LG cs.CVcited by 50

On the map: Diffusion models → Robustness, privacy and adversarial aspects

also in 1 more
Abstract (arXiv)

Diffusion models excel at generating visually striking content from text but can inadvertently produce undesirable or harmful content when trained on unfiltered internet data. A practical solution is to selectively removing target concepts from the model, but this may impact the remaining concepts. Prior approaches have tried to balance this by introducing a loss term to preserve neutral content or a regularization term to minimize changes in the model parameters, yet resolving this trade-off remains challenging. In this work, we propose to identify and preserving concepts most affected by parameter changes, termed as textitadversarial concepts. This approach ensures stable erasure with minimal impact on the other concepts. We demonstrate the effectiveness of our method using the Stable Diffusion model, showing that it outperforms state-of-the-art erasure methods in eliminating unwanted content while maintaining the integrity of other unrelated elements. Our code is available at https://github.com/tuananhbui89/Erasing-Adversarial-Preservation.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 35 of its 42 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

35 of 42 references.

Closest ancestors · 17

Same direction, newest first

  1. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models24 May
  2. 2024Hiding and Recovering Knowledge in Text-to-Image Diffusion Models via Learnable Prompts18 Mar
  3. 2024MACE: Mass Concept Erasure in Diffusion Models10 Mar
  4. 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications26 Dec
  5. 2023MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learning22 Nov
  6. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now18 Oct
  7. 2023Toward effective protection against diffusion based mimicry through score distillation2 Oct
  8. 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts12 Sep
  9. 2023Unified Concept Editing in Diffusion Models25 Aug
  10. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models23 May
  11. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Models20 May
  12. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models30 Mar
5 more
  1. 2023Anti-DreamBooth: Protecting users from personalized text-to-image synthesis27 Mar
  2. 2023Ablating Concepts in Text-to-Image Diffusion Models23 Mar
  3. 2023Erasing Concepts from Diffusion Models13 Mar
  4. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models9 Nov
  5. 2022Red-Teaming the Stable Diffusion Safety Filter3 Oct

Other references · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  11. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  12. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
6 more
  1. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  2. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
  3. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 130
  4. 2023EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protectioncited by 111
  5. 2022A Unified Wasserstein Distributional Robustness Framework for Adversarial Trainingcited by 36
  6. 2020Improving Adversarial Robustness by Enforcing Local and Global Compactnesscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 27; 2026: 22.

50 papers.

Sort:

By area:

  1. 2024EraseAnything: Enabling Concept Erasure in Rectified Flow Transformerscited by 95
  2. 2025Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Conceptscited by 62
  3. 2025Fantastic Targets for Concept Erasure in Diffusion Models and Where To Find Themcited by 37
  4. 2025Localized Concept Erasure for Text-to-Image Diffusion Models Using Training-Free Gated Low-Rank Adaptationcited by 33
  5. 2025One Image is Worth a Thousand Words: A Usability Preservable Text-Image Collaborative Erasing Frameworkcited by 16
  6. 2025Minimalist Concept Erasure in Generative Modelscited by 13
  7. 2025Side Effects of Erasing Concepts from Diffusion Modelscited by 13
  8. 2025Revoking Amnesia: RL-based Trajectory Optimization to Resurrect Erased Concepts in Diffusion Modelscited by 11
  9. 2025Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppressioncited by 10
  10. 2025Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspectivecited by 10
  11. 2025Erased but Not Forgotten: How Backdoors Compromise Concept Erasurecited by 9
  12. 2025Sparse Autoencoder as a Zero-Shot Classifier for Concept Erasing in Text-to-Image Diffusion Modelscited by 8
38 more
  1. 2025SuMa: A Subspace Mapping Approach for Robust and Effective Concept Erasure in Text-to-Image Diffusion Modelscited by 8
  2. 2025Semantic Surgery: Zero-Shot Concept Erasure in Diffusion Modelscited by 8
  3. 2025Concept Steerers: Leveraging K-Sparse Autoencoders for Test-Time Controllable Generationscited by 7
  4. 2026Differential Vector Erasure: Unified Training-Free Concept Erasure for Flow Matching Modelscited by 5
  5. 2026AEGIS: Adversarial Target-Guided Retention-Data-Free Robust Concept Erasure from Diffusion Modelscited by 5
  6. 2025EAR: Erasing Concepts from Unified Autoregressive Modelscited by 4
  7. 2025Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocolscited by 4
  8. 2026Z-Erase: Enabling Concept Erasure in Single-Stream Diffusion Transformerscited by 4
  9. 2025ACE: Attentional Concept Erasure in Diffusion Modelscited by 3
  10. 2025Erased, But Not Forgotten: Erased Rectified Flow Transformers Still Remain Unsafe Under Concept Attackcited by 3
  11. 2025TRACE: Trajectory-Constrained Concept Erasure in Diffusion Modelscited by 2
  12. 2025VCE: Safe Autoregressive Image Generation via Visual Contrast Exploitationcited by 2
  13. 2025A Unified Framework for Diffusion Model Unlearning with f-Divergencecited by 2
  14. 2025GrOCE:Graph-Guided Online Concept Erasure for Text-to-Image Diffusion Modelscited by 2
  15. 2025From Unlearning to UNBRANDING: A Benchmark for Trademark-Safe Text-to-Image Generationcited by 2
  16. 2026MacPrompt: Maraconic-guided Jailbreak against Text-to-Image Modelscited by 2
  17. 2025Beyond Fixed Anchors: Precisely Erasing Concepts with Sibling Exclusive Counterpartscited by 1
  18. 2025Rethinking Robust Adversarial Concept Erasure in Diffusion Modelscited by 1
  19. 2025Unconsciously Forget: Mitigating Memorization; Without Knowing What is being Memorizedcited by 1
  20. 2026EraseAnything++: Enabling Concept Erasure in Rectified Flow Transformers Leveraging Multi-Object Optimizationcited by 1
  21. 2026Unlearning the Unpromptable: Prompt-free Instance Unlearning in Diffusion Modelscited by 1
  22. 2026Erasure or Erosion? Evaluating Compositional Degradation in Unlearned Text-To-Image Diffusion Modelscited by 1
  23. 2026SHIFT: Steering Hidden Intermediates in Flow Transformerscited by 1
  24. 2026Beyond Text Prompts: Precise Concept Erasure through Text-Image Collaborationcited by 1
  25. 2026Empty SPACE: Cross-Attention Sparsity for Concept Erasure in Diffusion Modelscited by 1
  26. 2026Semantic Steering for Controllable Generation: Tuning-Free Concept Erasure in Multimodal Diffusion Transformerscited by 1
  27. 2026Your Unlearning Gives You Away: Identifying Erased Concepts in Diffusion Modelscited by 1
  28. 2025Mitigating Semantic Collapse in Generative Personalization with Test-Time Embedding Adjustmentcited by 0
  29. 2026BARRIER: Bounded Activation Regions for Robust Information Erasurecited by 0
  30. 2026Concept Unlearning via Cross-Attention Activation Projection for Diffusion Modelscited by 0
  31. 2026CoreUnlearn: Rethinking Concept Unlearning through Disentangled Component-Level Erasure in Text-guided Diffusion Modelscited by 0
  32. 2026SACE: Concept Erasure at the Semantic Singularity in Visual Autoregressive Modelscited by 0
  33. 2026AutoAnchor: Stable Diffusion Unlearning Using Cross-Attention as a Manifold Surrogatecited by 0
  34. 2026Stress Testing Concept Erasure with Large Language Model Agentscited by 0
  35. 2026Erase but Preserve: Controllable Removal of Copyrighted Animation Characters via Optimized Semantic Anchorscited by 0
  36. 2026Concept Score Relearning: A Unified Cross-Architecture Attack on Concept Erasurecited by 0
  37. 2026You Can't Have It Both Ways: Concept Entanglement Limits Diffusion Model Unlearningcited by 0
  38. 2026Weeding Out Bad Seeds: Initial-Noise-Robust Unlearning for Text-to-Image Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025When Are Concepts Erased From Diffusion Models?
  2. 2025Rethinking the Vulnerability of Concept Erasure and a New Method
  3. 2025A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Models
  4. 2024Pruning for Robust Concept Erasing in Diffusion Models
  5. 2025FADE: Adversarial Concept Erasure in Flow Models
  6. 2025Robust Concept Erasure in Diffusion Models: A Theoretical Perspective on Security and Robustness
  7. 2024Unlearning or Concealment? A Critical Analysis and Evaluation Metrics for Unlearning in Diffusion Models
  8. 2026No Concept Escapes the Audit: Auditing-Aware Unlearning for Verifiable Concept Erasure in Diffusion Models