# Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradient

arXiv: https://arxiv.org/abs/2405.15304 · HTML: https://mlsanity.com/paper/2405.15304/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 21 of its 54 references that are in the map.
- First posted on arXiv 24 May 2024.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)
- LLM safety and security / [Vision-language and text-to-image safety](https://mlsanity.com/safety/vision-language-and-text-to-image-safety/)

## Builds on (most cited first, 21 of 21)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models](https://mlsanity.com/paper/2211.05105/) (2022)
- [Unified Concept Editing in Diffusion Models](https://mlsanity.com/paper/2308.14761/) (2023)
- [Prompt-aligned Gradient for Prompt Tuning](https://mlsanity.com/paper/2205.14865/) (2022)
- [To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now](https://mlsanity.com/paper/2310.11868/) (2023)
- [One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications](https://mlsanity.com/paper/2312.16145/) (2023)
- [Membership Inference Attacks Against Text-to-image Generation Models](https://mlsanity.com/paper/2210.00968/) (2022)
- [Generated Faces in the Wild: Quantitative Comparison of Stable Diffusion, Midjourney and DALL-E 2](https://mlsanity.com/paper/2210.00586/) (2022)
- [Endowing Pre-trained Graph Models with Provable Fairness](https://mlsanity.com/paper/2402.12161/) (2024)
- [Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior](https://mlsanity.com/paper/2310.11284/) (2023)

## Built on by (most cited first, 30 of 69)

- [SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation](https://mlsanity.com/paper/2410.12761/) (2024)
- [Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety](https://mlsanity.com/paper/2502.05206/) (2025)
- [Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Concepts](https://mlsanity.com/paper/2410.12777/) (2024)
- [SPEED: Scalable, Precise, and Efficient Concept Erasure for Diffusion Models](https://mlsanity.com/paper/2503.07392/) (2025)
- [Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Survey](https://mlsanity.com/paper/2407.15861/) (2024)
- [Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey](https://mlsanity.com/paper/2408.03400/) (2024)
- [Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey](https://mlsanity.com/paper/2411.09259/) (2024)
- [CURE: Concept Unlearning via Orthogonal Representation Editing in Diffusion Models](https://mlsanity.com/paper/2505.12677/) (2025)
- [Minimalist Concept Erasure in Generative Models](https://mlsanity.com/paper/2507.13386/) (2025)
- [On the Limitations and Prospects of Machine Unlearning for Generative AI](https://mlsanity.com/paper/2408.00376/) (2024)
- [Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppression](https://mlsanity.com/paper/2505.19398/) (2025)
- [Image Can Bring Your Memory Back: A Novel Multi-Modal Guided Attack against Image Generation Model Unlearning](https://mlsanity.com/paper/2507.07139/) (2025)
- [Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspective](https://mlsanity.com/paper/2511.07970/) (2025)
- [CE-SDWV: Effective and Efficient Concept Erasure for Text-to-Image Diffusion Models via a Semantic-Driven Word Vocabulary](https://mlsanity.com/paper/2501.15562/) (2025)
- [Model Integrity when Unlearning with T2I Diffusion Models](https://mlsanity.com/paper/2411.02068/) (2024)
- [AEIOU: A Unified Defense Framework against NSFW Prompts in Text-to-Image Models](https://mlsanity.com/paper/2412.18123/) (2024)
- [Position: Bridge the Gaps between Machine Unlearning and AI Regulation](https://mlsanity.com/paper/2502.12430/) (2025)
- [TarPro: Targeted Protection against Malicious Image Editing](https://mlsanity.com/paper/2503.13994/) (2025)
- [Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation](https://mlsanity.com/paper/2602.24041/) (2026)
- [OrthoEraser: Coupled-Neuron Orthogonal Projection for Concept Erasure](https://mlsanity.com/paper/2603.11493/) (2026)
- [Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks](https://mlsanity.com/paper/2607.07907/) (2026)
- [Plug-and-Play Interpretable Responsible Text-to-Image Generation via Dual-Space Multi-facet Concept Control](https://mlsanity.com/paper/2503.18324/) (2025)
- [Efficient Utility-Preserving Machine Unlearning with Implicit Gradient Surgery](https://mlsanity.com/paper/2510.22124/) (2025)
- [PurifyGen: A Risk-Discrimination and Semantic-Purification Model for Safe Text-to-Image Generation](https://mlsanity.com/paper/2512.23546/) (2025)
- [Rethinking Byzantine Robustness in Federated Recommendation from Sparse Aggregation Perspective](https://mlsanity.com/paper/2501.03301/) (2025)
- [Concept Lancet: Image Editing with Compositional Representation Transplant](https://mlsanity.com/paper/2504.02828/) (2025)
- [Now You See It, Now You Don't - Instant Concept Erasure for Safe Text-to-Image and Video Generation](https://mlsanity.com/paper/2511.18684/) (2025)
- [Wukong Framework for Not Safe For Work Detection in Text-to-Image systems](https://mlsanity.com/paper/2508.00591/) (2025)
- [Collapse of Irrelevant Representations (CIR) Ensures Robust and Non-Disruptive LLM Unlearning](https://mlsanity.com/paper/2509.11816/) (2025)
- [Hawk: Leveraging Spatial Context for Faster Autoregressive Text-to-Image Generation](https://mlsanity.com/paper/2510.25739/) (2025)

## Abstract (arXiv)

Text-to-image diffusion models have achieved remarkable success in generating photorealistic images. However, the inclusion of sensitive information during pre-training poses significant risks. Machine Unlearning (MU) offers a promising solution to eliminate sensitive concepts from these models. Despite its potential, existing MU methods face two main challenges: 1) limited generalization, where concept erasure is effective only within the unlearned set, failing to prevent sensitive concept generation from out-of-set prompts; and 2) utility degradation, where removing target concepts significantly impacts the model's overall performance. To address these issues, we propose a novel concept domain correction framework named DoCo (Domain Correction). By aligning the output domains of sensitive and anchor concepts through adversarial training, our approach ensures comprehensive unlearning of target concepts. Additionally, we introduce a concept-preserving gradient surgery technique that mitigates conflicting gradient components, thereby preserving the model's utility while unlearning specific concepts. Extensive experiments across various instances, styles, and offensive concepts demonstrate the effectiveness of our method in unlearning targeted concepts with minimal impact on related concepts, outperforming previous approaches even for out-of-distribution prompts.