# Unified Concept Editing in Diffusion Models

arXiv: https://arxiv.org/abs/2308.14761 · HTML: https://mlsanity.com/paper/2308.14761/

## Key facts

- Cited by 388 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 34 of its 52 references that are in the map.
- First posted on arXiv 25 Aug 2023.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Erasing Concepts from Diffusion Models](https://mlsanity.com/paper/2303.07345/) (2023)
- [Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models](https://mlsanity.com/paper/2211.05105/) (2022)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [Ablating Concepts in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.13516/) (2023)
- [Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.17591/) (2023)
- [Red-Teaming the Stable Diffusion Safety Filter](https://mlsanity.com/paper/2210.04610/) (2022)
- [Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale](https://mlsanity.com/paper/2211.03759/) (2022)
- [DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models](https://mlsanity.com/paper/2202.04053/) (2022)
- [Stable Bias: Analyzing Societal Representations in Diffusion Models](https://mlsanity.com/paper/2303.11408/) (2023)
- [Understanding and Mitigating Copying in Diffusion Models](https://mlsanity.com/paper/2305.20086/) (2023)
- [Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models](https://mlsanity.com/paper/2302.04222/) (2023)
- [fastai: A Layered API for Deep Learning](https://mlsanity.com/paper/2002.04688/) (2020)
- [Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models](https://mlsanity.com/paper/2305.10120/) (2023)
- [Raising the Cost of Malicious AI-Powered Image Editing](https://mlsanity.com/paper/2302.06588/) (2023)
- [Debiasing Vision-Language Models via Biased Prompts](https://mlsanity.com/paper/2302.00070/) (2023)
- [The Privacy Onion Effect: Memorization is Relative](https://mlsanity.com/paper/2206.10469/) (2022)
- [Editing Implicit Assumptions in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.08084/) (2023)
- [Rewriting a Deep Generative Model](https://mlsanity.com/paper/2007.15646/) (2020)

## Built on by (most cited first, 30 of 388)

- [MACE: Mass Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2403.06135/) (2024)
- [To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now](https://mlsanity.com/paper/2310.11868/) (2023)
- [Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2405.15234/) (2024)
- [Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models](https://mlsanity.com/paper/2311.12092/) (2023)
- [Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models](https://mlsanity.com/paper/2407.12383/) (2024)
- [SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation](https://mlsanity.com/paper/2410.12761/) (2024)
- [Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers](https://mlsanity.com/paper/2311.17717/) (2023)
- [EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers](https://mlsanity.com/paper/2412.20413/) (2024)
- [Finetuning Text-to-Image Diffusion Models for Fairness](https://mlsanity.com/paper/2311.07604/) (2023)
- [SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders](https://mlsanity.com/paper/2501.18052/) (2025)
- [Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models](https://mlsanity.com/paper/2404.02928/) (2024)
- [Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation](https://mlsanity.com/paper/2311.17216/) (2023)
- [Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradient](https://mlsanity.com/paper/2405.15304/) (2024)
- [Toward effective protection against diffusion based mimicry through score distillation](https://mlsanity.com/paper/2311.12832/) (2023)
- [Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Concepts](https://mlsanity.com/paper/2504.12782/) (2025)
- [OpenBias: Open-set Bias Detection in Text-to-Image Generative Models](https://mlsanity.com/paper/2404.07990/) (2024)
- [Erasing Undesirable Influence in Diffusion Models](https://mlsanity.com/paper/2401.05779/) (2024)
- [ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning](https://mlsanity.com/paper/2405.19237/) (2024)
- [Scissorhands: Scrub Data Influence via Connection Sensitivity in Networks](https://mlsanity.com/paper/2401.06187/) (2024)
- [UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Models](https://mlsanity.com/paper/2402.11846/) (2024)
- [Direct Unlearning Optimization for Robust and Safe Text-to-Image Models](https://mlsanity.com/paper/2407.21035/) (2024)
- [Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservation](https://mlsanity.com/paper/2410.15618/) (2024)
- [Machine Unlearning in Generative AI: A Survey](https://mlsanity.com/paper/2407.20516/) (2024)
- [T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation](https://mlsanity.com/paper/2501.12612/) (2025)
- [Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning](https://mlsanity.com/paper/2403.07362/) (2024)
- [Dissecting Language Models: Machine Unlearning via Selective Pruning](https://mlsanity.com/paper/2403.01267/) (2024)
- [STEREO: A Two-Stage Framework for Adversarially Robust Concept Erasing from Text-to-Image Diffusion Models](https://mlsanity.com/paper/2408.16807/) (2024)
- [Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Concepts](https://mlsanity.com/paper/2410.12777/) (2024)
- [Precise, Fast, and Low-cost Concept Erasure in Value Space: Orthogonal Complement Matters](https://mlsanity.com/paper/2412.06143/) (2024)
- [Finding NeMo: Localizing Neurons Responsible For Memorization in Diffusion Models](https://mlsanity.com/paper/2406.02366/) (2024)

## Abstract (arXiv)

Text-to-image models suffer from various safety issues that may limit their suitability for deployment. Previous methods have separately addressed individual issues of bias, copyright, and offensive content in text-to-image models. However, in the real world, all of these issues appear simultaneously in the same model. We present a method that tackles all issues with a single approach. Our method, Unified Concept Editing (UCE), edits the model without training using a closed-form solution, and scales seamlessly to concurrent edits on text-conditional diffusion models. We demonstrate scalable simultaneous debiasing, style erasure, and content moderation by editing text-to-image projections, and we present extensive experiments demonstrating improved efficacy and scalability over prior work. Our code is available at https://unified.baulab.info