Erased but Not Forgotten: How Backdoors Compromise Concept Erasure
On the map: Diffusion models → Robustness, privacy and adversarial aspects
Abstract (arXiv)
The expansion of text-to-image diffusion models has raised concerns about harmful outputs, from fabricated depictions of public figures to sexually explicit imagery. To mitigate such risks, prior work has proposed concept erasure methods that aim to sever unwanted concepts from the model via fine-tuning, yet it remains unclear whether these approaches truly remove all links to the harmful concept or merely conceal superficial connections. In this work, we reveal a critical vulnerability, the Erasure Evasion Backdoor (EEB): an adversary binds a backdoor trigger to a concept slated for removal, and this malicious link survives subsequent erasure. We show that both black-box and white-box adversaries can instantiate this threat. Across six state-of-the-art erasure methods, including robust ones that explicitly search for alternative representations of the target concept, EEB consistently exposes harmful content: up to 82% success against celebrity-identity unlearning, up to 94% for object erasure, and up to 16 times amplification of explicit-content exposure. While EEB uncovers a blind spot in current erasure methods, it also provides a diagnostic tool for stress-testing future concept erasure techniques. Our code is available at https://github.com/multimodal-ai-lab/EEB.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 79 of its 109 references that are in the map. First posted on arXiv 29 Apr 2025.
Builds on
79 of 109 references.
Closest ancestors · 29
Same direction, newest first
- 2025Fantastic Targets for Concept Erasure in Diffusion Models and Where To Find Them31 Jan
- 2025ACE: Anti-Editing Concept Erasure in Text-to-Image Models3 Jan
- 2024Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimization5 Dec
- 2024Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservation21 Oct
- 2024Unstable Unlearning: The Hidden Risk of Concept Resurgence in Diffusion Models10 Oct
- 2024STEREO: A Two-Stage Framework for Adversarially Robust Concept Erasing from Text-to-Image Diffusion Models29 Aug
- 2024Safeguard Text-to-Image Diffusion Models with Human Feedback Inversion17 Jul
- 2024Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models17 Jul
- 2024T2IShield: Defending Against Backdoors on Text-to-Image Diffusion Models5 Jul
- 2024Backdooring Bias (B2) into Stable Diffusion Models21 Jun
- 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models24 May
- 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Models10 Apr
17 more
- 2024MACE: Mass Concept Erasure in Diffusion Models10 Mar
- 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications26 Dec
- 2023Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers29 Nov
- 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now18 Oct
- 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts12 Sep
- 2023Unified Concept Editing in Diffusion Models25 Aug
- 2023Degeneration-Tuning: Using Scrambled Grid shield Unwanted Concepts from Stable Diffusion2 Aug
- 2023Towards Safe Self-Distillation of Internet-Scale Text-to-Image Diffusion Models12 Jul
- 2023VillanDiffusion: A Unified Backdoor Attack Framework for Diffusion Models12 Jun
- 2023Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models18 May
- 2023Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning7 May
- 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models30 Mar
- 2023Ablating Concepts in Text-to-Image Diffusion Models23 Mar
- 2023Erasing Concepts from Diffusion Models13 Mar
- 2022How to Backdoor Diffusion Models?11 Dec
- 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models9 Nov
- 2022Red-Teaming the Stable Diffusion Safety Filter3 Oct
Other references · 50
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Classifier-Free Diffusion Guidancecited by 5,717
38 more
- 2013Intriguing properties of neural networkscited by 5,114
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
- 2020Knowledge Distillation: A Surveycited by 1,578
- 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
- 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
- 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
- 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
- 2021Membership Inference Attacks From First Principlescited by 839
- 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
- 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
- 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
- 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
- 2021GFlowNet Foundationscited by 286
- 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
- 2023Poisoning Language Models During Instruction Tuningcited by 254
- 2020The Creation and Detection of Deepfakes: A Surveycited by 246
- 2020Media Forensics and DeepFakes: an overviewcited by 208
- 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 175
- 2020Backdoor Attacks Against Deep Learning Systems in the Physical Worldcited by 129
- 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 128
- 2023Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Modelscited by 87
- 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 81
- 2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 72
- 2022Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesiscited by 59
- 2024Direct Unlearning Optimization for Robust and Safe Text-to-Image Modelscited by 50
- 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Modelscited by 40
- 2022Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attackscited by 37
- 2023BAGM: A Backdoor Attack for Manipulating Text-to-Image Generative Modelscited by 37
- 2023Stealthy Backdoor Attack for Code Modelscited by 32
- 2022TrojViT: Trojan Insertion in Vision Transformerscited by 30
- 2024AlignGuard: Scalable Safety Alignment for Text-to-Image Generationcited by 17
- 2024Targeted Unlearning with Single Layer Unlearning Gradientcited by 16
- 2025Erasing More Than Intended? How Concept Erasure Degrades the Generation of Non-Target Conceptscited by 12
- 2023Exploiting Machine Unlearning for Backdoor Attacks in Deep Learning Systemcited by 4
- 2025ReVeil: Unconstrained Concealed Backdoor Attack on Deep Neural Networks using Machine Unlearningcited by 3
Built on by
Citing papers per year: 2025: 3; 2026: 6.
9 papers.
- 2025Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearningcited by 4
- 2025Memory Self-Regeneration: Uncovering Hidden Knowledge in Unlearned Modelscited by 2
- 2026GEM: Geometric Erasure by Contrastive Velocity Matching in Rectified Flowscited by 2
- 2025From Unlearning to UNBRANDING: A Benchmark for Trademark-Safe Text-to-Image Generationcited by 1
- 2026Steerability of Instrumental-Convergence Tendencies in LLMscited by 1
- 2026Obliviate: Erasing Concepts from Autoregressive Image Generation Modelscited by 1
- 2026Tuning Just Enough: Lightweight Backdoor Attacks on Multi-Encoder Diffusion Modelscited by 0
- 2026Whispers in the Noise: Surrogate-Guided Concept Awakening via a Multi-Agent Frameworkcited by 0
- 2026Token by Token, Compromised: Backdoor Vulnerabilities in Unified Autoregressive Modelscited by 0
- 2026Obliviate: Erasing Concepts from Autoregressive Image Generation Models26 Jun 2026
- 2026GEM: Geometric Erasure by Contrastive Velocity Matching in Rectified Flows29 May 2026
- 2026Token by Token, Compromised: Backdoor Vulnerabilities in Unified Autoregressive Models19 May 2026
- 2026Whispers in the Noise: Surrogate-Guided Concept Awakening via a Multi-Agent Framework18 May 2026
- 2026Tuning Just Enough: Lightweight Backdoor Attacks on Multi-Encoder Diffusion Models4 Mar 2026
- 2026Steerability of Instrumental-Convergence Tendencies in LLMs4 Jan 2026
- 2025From Unlearning to UNBRANDING: A Benchmark for Trademark-Safe Text-to-Image Generation15 Dec 2025
- 2025Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning19 Oct 2025
- 2025Memory Self-Regeneration: Uncovering Hidden Knowledge in Unlearned Models26 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2025Robust Concept Erasure in Diffusion Models: A Theoretical Perspective on Security and Robustness
- 2025Rethinking the Vulnerability of Concept Erasure and a New Method
- 2024Probing Unlearned Diffusion Models: A Transferable Adversarial Attack Perspective
- 2025M-ErasureBench: A Comprehensive Multimodal Evaluation Benchmark for Concept Erasure in Diffusion Models
- 2025A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Models
- 2025Comprehensive Evaluation and Analysis for NSFW Concept Erasure in Text-to-Image Diffusion Models
- 2025Comprehensive Assessment and Analysis for NSFW Content Erasure in Text-to-Image Diffusion Models
- 2025VideoEraser: Concept Erasure in Text-to-Video Diffusion Models