mlsanityData to 7 Oct 2026

Unstable Unlearning: The Hidden Risk of Concept Resurgence in Diffusion Models

arXiv 2410.08074 ↗First posted 10 Oct 2024cs.LG cs.CR cs.CVcited by 14

On the map: Diffusion models → Robustness, privacy and adversarial aspects

also in 1 more
Abstract (arXiv)

Text-to-image diffusion models rely on massive, web-scale datasets. Training them from scratch is computationally expensive, and as a result, developers often prefer to make incremental updates to existing models. These updates often compose fine-tuning steps (to learn new concepts or improve model performance) with "unlearning" steps (to "forget" existing concepts, such as copyrighted works or explicit content). In this work, we demonstrate a critical and previously unknown vulnerability that arises in this paradigm: even under benign, non-adversarial conditions, fine-tuning a text-to-image diffusion model on seemingly unrelated images can cause it to "relearn" concepts that were previously "unlearned." We comprehensively investigate the causes and scope of this phenomenon, which we term concept resurgence, by performing a series of experiments which compose "concept unlearning" with subsequent fine-tuning of Stable Diffusion v1.4 and Stable Diffusion v2.1. Our findings underscore the fragility of composing incremental model updates, and raise serious new concerns about current approaches to ensuring the safety and alignment of text-to-image diffusion models.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 42 of its 67 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

42 of 67 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models17 Jul
  2. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models24 May
  3. 2024Erasing Concepts from Text-to-Image Diffusion Models with Few-shot Unlearning12 May
  4. 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models2 Apr
  5. 2024MACE: Mass Concept Erasure in Diffusion Models10 Mar
  6. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation19 Oct
  7. 2023On Copyright Risks of Text-to-Image Diffusion Models15 Sep
  8. 2023Unified Concept Editing in Diffusion Models25 Aug
  9. 2023Towards Safe Self-Distillation of Internet-Scale Text-to-Image Diffusion Models12 Jul
  10. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models23 May
  11. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Models20 May
  12. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models30 Mar
4 more
  1. 2023Erasing Concepts from Diffusion Models13 Mar
  2. 2023Extracting Training Data from Diffusion Models30 Jan
  3. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models9 Nov
  4. 2022Red-Teaming the Stable Diffusion Safety Filter3 Oct

Other references · 26

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  7. 2019Machine Unlearningcited by 1,020
  8. 2019Large Scale Incremental Learningcited by 727
  9. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  10. 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 360
  11. 2022A Survey of Machine Unlearningcited by 298
  12. 2023Towards Unbounded Machine Unlearningcited by 261
14 more
  1. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  2. 2023Understanding and Mitigating Copying in Diffusion Modelscited by 205
  3. 2020When Machine Unlearning Jeopardizes Privacycited by 181
  4. 2021Adaptive Machine Unlearningcited by 164
  5. 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 150
  6. 2022The Privacy Onion Effect: Memorization is Relativecited by 128
  7. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 113
  8. 2021Talk-to-Edit: Fine-Grained Facial Editing via Dialogcited by 102
  9. 2024Scissorhands: Scrub Data Influence via Connection Sensitivity in Networkscited by 50
  10. 2021Hard to Forget: Poisoning Attacks on Certified Machine Unlearningcited by 47
  11. 2022Algorithms that Approximate Data Removal: New Results and Limitationscited by 39
  12. 2022Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attackscited by 37
  13. 2023Ticketed Learning-Unlearning Schemescited by 10
  14. 2023Measuring the Success of Diffusion Models at Imitating Human Artistscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 8.

14 papers.

Sort:

By area:

  1. 2025ForgetMe: Evaluating Selective Forgetting in Generative Modelscited by 11
  2. 2025Video Unlearning via Low-Rank Refusal Vectorcited by 11
  3. 2025Erased but Not Forgotten: How Backdoors Compromise Concept Erasurecited by 9
  4. 2025SAEmnesia: Erasing Concepts in Diffusion Models with Supervised Sparse Autoencoderscited by 8
  5. 2026Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarkscited by 6
  6. 2026An Illusion of Unlearning? Assessing Machine Unlearning Through Internal Representationscited by 3
  7. 2025SPQR: A Multi-Dimensional Benchmark for Safety Alignment under Benign Model Adaptationcited by 2
  8. 2026The Illusion of Forgetting: Attack Unlearned Diffusion via Initial Latent Variable Optimizationcited by 1
  9. 2026Evaluation without Generation: Non-Generative Assessment of Harmful Model Specialization with Applications to CSAMcited by 1
  10. 2025Beyond Memorization: Selective Learning for Copyright-Safe Diffusion Model Trainingcited by 0
  11. 2026Privacy in Image Datasets: A Case Study on Pregnancy Ultrasoundscited by 0
  12. 2026Compensation-free Machine Unlearning in Text-to-Image Diffusion Models by Eliminating the Mutual Informationcited by 0
2 more
  1. 2026Mitigating Sequential Reappearance in Diffusion Data-Point Unlearningcited by 0
  2. 2026Continual Data Unlearning in Diffusion Models via Transition-based Regularizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Forget Many, Forget Right: Scalable and Precise Concept Unlearning in Diffusion Models
  2. 2024Unlearning or Concealment? A Critical Analysis and Evaluation Metrics for Unlearning in Diffusion Models
  3. 2026eval-unlearn: Benchmarking unlearning in Text-to-Image Diffusion Models
  4. 2024Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradient
  5. 2026Projected Gradient Unlearning for Text-to-Image Diffusion Models: Defending Against Concept Revival Attacks
  6. 2026A Concept is More Than a Word: Diversified Unlearning in Text-to-Image Diffusion Models
  7. 2025Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspective
  8. 2026Weeding Out Bad Seeds: Initial-Noise-Robust Unlearning for Text-to-Image Diffusion Models