# Editing Massive Concepts in Text-to-Image Diffusion Models arXiv: https://arxiv.org/abs/2403.13807 ยท HTML: https://mlsanity.com/paper/2403.13807/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. - Builds on 27 of its 35 references that are in the map. - First posted on arXiv 20 Mar 2024. ## Placed in - Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/) ## Builds on (most cited first, 27 of 27) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022) - [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022) - [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023) - [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021) - [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022) - [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018) - [Erasing Concepts from Diffusion Models](https://mlsanity.com/paper/2303.07345/) (2023) - [Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models](https://mlsanity.com/paper/2211.05105/) (2022) - [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022) - [Unified Concept Editing in Diffusion Models](https://mlsanity.com/paper/2308.14761/) (2023) - [Ablating Concepts in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.13516/) (2023) - [Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.17591/) (2023) - [DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models](https://mlsanity.com/paper/2202.04053/) (2022) - [Stable Bias: Analyzing Societal Representations in Diffusion Models](https://mlsanity.com/paper/2303.11408/) (2023) - [Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models](https://mlsanity.com/paper/2302.04222/) (2023) - [Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models](https://mlsanity.com/paper/2305.10120/) (2023) - [Editing Implicit Assumptions in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.08084/) (2023) - [Towards Safe Self-Distillation of Internet-Scale Text-to-Image Diffusion Models](https://mlsanity.com/paper/2307.05977/) (2023) - [ReFACT: Updating Text-to-Image Models by Editing the Text Encoder](https://mlsanity.com/paper/2306.00738/) (2023) ## Built on by (most cited first, 15 of 15) - [SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation](https://mlsanity.com/paper/2410.12761/) (2024) - [Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Concepts](https://mlsanity.com/paper/2504.12782/) (2025) - [CURE: Concept Unlearning via Orthogonal Representation Editing in Diffusion Models](https://mlsanity.com/paper/2505.12677/) (2025) - [Replication in Visual Diffusion Models: A Survey and Outlook](https://mlsanity.com/paper/2408.00001/) (2024) - [Unveiling Concept Attribution in Diffusion Models](https://mlsanity.com/paper/2412.02542/) (2024) - [Six-CD: Benchmarking Concept Removals for Benign Text-to-image Diffusion Models](https://mlsanity.com/paper/2406.14855/) (2024) - [A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2502.14896/) (2025) - [Plug-and-Play Interpretable Responsible Text-to-Image Generation via Dual-Space Multi-facet Concept Control](https://mlsanity.com/paper/2503.18324/) (2025) - [Hiding Images in Diffusion Models by Editing Learned Score Functions](https://mlsanity.com/paper/2503.18459/) (2025) - [Implicit Priors Editing in Stable Diffusion via Targeted Token Adjustment](https://mlsanity.com/paper/2412.03400/) (2024) - [Do Concept Replacement Techniques Really Erase Unacceptable Concepts?](https://mlsanity.com/paper/2506.08991/) (2025) - [How Robust is Model Editing after Fine-Tuning? An Empirical Study on Text-to-Image Diffusion Models](https://mlsanity.com/paper/2506.18428/) (2025) - [Beyond Fixed Anchors: Precisely Erasing Concepts with Sibling Exclusive Counterparts](https://mlsanity.com/paper/2510.16342/) (2025) - [Empty SPACE: Cross-Attention Sparsity for Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2605.10198/) (2026) - [LU-500: A Logo Benchmark for Concept Unlearning](https://mlsanity.com/paper/2607.24101/) (2026) ## Abstract (arXiv) Text-to-image diffusion models suffer from the risk of generating outdated, copyrighted, incorrect, and biased content. While previous methods have mitigated the issues on a small scale, it is essential to handle them simultaneously in larger-scale real-world scenarios. We propose a two-stage method, Editing Massive Concepts In Diffusion Models (EMCID). The first stage performs memory optimization for each individual concept with dual self-distillation from text alignment loss and diffusion noise prediction loss. The second stage conducts massive concept editing with multi-layer, closed form model editing. We further propose a comprehensive benchmark, named ImageNet Concept Editing Benchmark (ICEB), for evaluating massive concept editing for T2I models with two subtasks, free-form prompts, massive concept categories, and extensive evaluation metrics. Extensive experiments conducted on our proposed benchmark and previous benchmarks demonstrate the superior scalability of EMCID for editing up to 1,000 concepts, providing a practical approach for fast adjustment and re-deployment of T2I diffusion models in real-world applications.