mlsanityData to 14 Oct 2026

Side Effects of Erasing Concepts from Diffusion Models

arXiv 2508.15124 ↗First posted 20 Aug 2025cs.LG cs.CVcited by 13

On the map: Diffusion models → Robustness, privacy and adversarial aspects

Abstract (arXiv)

Concerns about text-to-image (T2I) generative models infringing on privacy, copyright, and safety have led to the development of concept erasure techniques (CETs). The goal of an effective CET is to prohibit the generation of undesired "target" concepts specified by the user, while preserving the ability to synthesize high-quality images of other concepts. In this work, we demonstrate that concept erasure has side effects and CETs can be easily circumvented. For a comprehensive measurement of the robustness of CETs, we present the Side Effect Evaluation (SEE) benchmark that consists of hierarchical and compositional prompts describing objects and their attributes. The dataset and an automated evaluation pipeline quantify side effects of CETs across three aspects: impact on neighboring concepts, evasion of targets, and attribute leakage. Our experiments reveal that CETs can be circumvented by using superclass-subclass hierarchy, semantically similar prompts, and compositional variants of the target. We show that CETs suffer from attribute leakage and a counterintuitive phenomenon of attention concentration or dispersal. We release our benchmark and evaluation tools to aid future work on robust concept erasure.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 34 of its 46 references that are in the map. First posted on arXiv 20 Aug 2025.

Builds on

34 of 46 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservation21 Oct
  2. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models24 May
  3. 2024MACE: Mass Concept Erasure in Diffusion Models10 Mar
  4. 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications26 Dec
  5. 2023Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers29 Nov
  6. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now18 Oct
  7. 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts12 Sep
  8. 2023Unified Concept Editing in Diffusion Models25 Aug
  9. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models9 Nov
  10. 2022Red-Teaming the Stable Diffusion Safety Filter3 Oct

Other references · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2019Machine Unlearningcited by 1,061
  7. 2022Mass-Editing Memory in a Transformercited by 1,001
  8. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 618
  9. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  10. 2021Fast Model Editing at Scalecited by 498
  11. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 465
  12. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
12 more
  1. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 311
  2. 2020Approximate Data Deletion from Machine Learning Modelscited by 255
  3. 2021Machine Unlearning of Features and Labelscited by 229
  4. 2022Versatile Diffusion: Text, Images and Variations All in One Diffusion Modelcited by 199
  5. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignmentcited by 171
  6. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearningcited by 104
  7. 2023Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editingcited by 76
  8. 2023ReFACT: Updating Text-to-Image Models by Editing the Text Encodercited by 32
  9. 2024Florence-VL: Enhancing Vision-Language Models with Generative Vision Encoder and Depth-Breadth Fusioncited by 18
  10. 2025Erasing More Than Intended? How Concept Erasure Degrades the Generation of Non-Target Conceptscited by 12
  11. 2024Rethinking Artistic Copyright Infringements in the Era of Text-to-Image Generative Modelscited by 9
  12. 2024Attention Overlap Is Responsible for The Entity Missing Problem in Text-to-image Diffusion Models!cited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 10.

13 papers.

Sort:

By area:

  1. 2026TraceRouter: Robust Safety for Large Foundation Models via Path-Level Interventioncited by 11
  2. 2026OrthoEraser: Coupled-Neuron Orthogonal Projection for Concept Erasurecited by 7
  3. 2025Locality-Aware Continual Unlearning for Diffusion Modelscited by 5
  4. 2025EMMA: Concept Erasure Benchmark with Comprehensive Semantic Metrics and Diverse Categoriescited by 4
  5. 2026Conditioned Activation Transport for T2I Safety Steeringcited by 3
  6. 2026To Erase, or Not to Erase: Robust Training-Free Concept Erasure with Preservation aware Adaptive Ranked Subspace Expansioncited by 3
  7. 2025Beyond the Safety Tax: Mitigating Unsafe Text-to-Image Generation via External Safety Rectificationcited by 2
  8. 2026OracleZoom: On-Policy Self-Distillation Inspired Reference-Constrained Recursive Image Super Resolutioncited by 1
  9. 2026BEFORE THE FLIP: Measuring Hidden Score Shifts In Quantized Vision Language Models Before The Answer Changes for Visual Question Answeringcited by 1
  10. 20263DEditSafe: Defending 3D Editing Pipelines from Unsafe Generationcited by 0
  11. 2026New Evidence, Same Choice: Testing Physical Experiment Selection in Vision Language Modelscited by 0
  12. 2026Motion Concept Unlearning in Video Diffusion Modelscited by 0
1 more
  1. 2026Compositional Concept Erasure in Text-to-Image Diffusion Models via Hierarchically Grounded Semantic Surgerycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Models
  2. 2025Erased or Dormant? Rethinking Concept Erasure Through Reversibility
  3. 2025M-ErasureBench: A Comprehensive Multimodal Evaluation Benchmark for Concept Erasure in Diffusion Models
  4. 2025Concept Corrector: Erase concepts on the fly for text-to-image diffusion models
  5. 2025Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppression
  6. 2026Localized Concept Erasure in Text-to-Image Diffusion Models via High-Level Representation Misdirection
  7. 2025CRCE: Coreference-Retention Concept Erasure in Text-to-Image Diffusion Models
  8. 2025Bi-Erasing: A Bidirectional Framework for Concept Removal in Diffusion Models