mlsanityData to 7 Oct 2026

MACE: Mass Concept Erasure in Diffusion Models

arXiv 2403.06135 ↗First posted 10 Mar 2024cs.CV cs.AI cs.LGcited by 298

On the map: Diffusion models → Robustness, privacy and adversarial aspects

Abstract (arXiv)

The rapid expansion of large-scale text-to-image diffusion models has raised growing concerns regarding their potential misuse in creating harmful or misleading content. In this paper, we introduce MACE, a finetuning framework for the task of mass concept erasure. This task aims to prevent models from generating images that embody unwanted concepts when prompted. Existing concept erasure methods are typically restricted to handling fewer than five concepts simultaneously and struggle to find a balance between erasing concept synonyms (generality) and maintaining unrelated concepts (specificity). In contrast, MACE differs by successfully scaling the erasure scope up to 100 concepts and by achieving an effective balance between generality and specificity. This is achieved by leveraging closed-form cross-attention refinement along with LoRA finetuning, collectively eliminating the information of undesirable concepts. Furthermore, MACE integrates multiple LoRAs without mutual interference. We conduct extensive evaluations of MACE against prior methods across four different tasks: object erasure, celebrity erasure, explicit content erasure, and artistic style erasure. Our results reveal that MACE surpasses prior methods in all evaluated tasks. Code is available at https://github.com/Shilin-LU/MACE.

Key facts. Cited by 298 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 59 of its 78 references that are in the map. First posted on arXiv 10 Mar 2024.

Builds on

59 of 78 references.

Closest ancestors · 12

Same direction, newest first

  1. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now18 Oct
  2. 2023Unified Concept Editing in Diffusion Models25 Aug
  3. 2023Degeneration-Tuning: Using Scrambled Grid shield Unwanted Concepts from Stable Diffusion2 Aug
  4. 2023Towards Safe Self-Distillation of Internet-Scale Text-to-Image Diffusion Models12 Jul
  5. 2023Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation2 Jun
  6. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models30 Mar
  7. 2023Ablating Concepts in Text-to-Image Diffusion Models23 Mar
  8. 2023Erasing Concepts from Diffusion Models13 Mar
  9. 2023Raising the Cost of Malicious AI-Powered Image Editing13 Feb
  10. 2023Extracting Training Data from Diffusion Models30 Jan
  11. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models9 Nov
  12. 2022Red-Teaming the Stable Diffusion Safety Filter3 Oct

Other references · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2020Denoising Diffusion Implicit Modelscited by 9,655
  10. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  11. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  12. 2023Segment Anythingcited by 8,158
35 more
  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  9. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  10. 2021Variational Diffusion Modelscited by 1,145
  11. 2022Mass-Editing Memory in a Transformercited by 976
  12. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  13. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  14. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  15. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  16. 2022Diffusion Models already have a Semantic Latent Spacecited by 324
  17. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  18. 2022Perception Prioritized Training of Diffusion Modelscited by 301
  19. 2020The Creation and Detection of Deepfakes: A Surveycited by 246
  20. 2020Media Forensics and DeepFakes: an overviewcited by 208
  21. 2023Glaze: Protecting Artists from Style Mimicry by Text-to-Image Modelscited by 200
  22. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
  23. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 175
  24. 2023Grounded Text-to-Image Synthesis with Attention Refocusingcited by 163
  25. 2022The Privacy Onion Effect: Memorization is Relativecited by 128
  26. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 128
  27. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
  28. 2020Rewriting a Deep Generative Modelcited by 105
  29. 2023Wavelet-based Fourier Information Interaction with Frequency Diffusion Adjustment for Underwater Image Restorationcited by 86
  30. 2023Spontaneous Symmetry Breaking in Generative Diffusion Modelscited by 74
  31. 2023Pyramid Diffusion Models For Low-light Image Enhancementcited by 67
  32. 2023Localizing and Editing Knowledge in Text-to-Image Generative Modelscited by 52
  33. 2023ReFACT: Updating Text-to-Image Models by Editing the Text Encodercited by 32
  34. 2024Learning A Physical-aware Diffusion Model Based on Transformer for Underwater Image Enhancementcited by 24
  35. 2023A Geometric Perspective on Diffusion Modelscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 44; 2025: 137; 2026: 117.

298 papers.

Sort:

By area:

  1. 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 125
  2. 2024Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advancescited by 105
  3. 2024EraseAnything: Enabling Concept Erasure in Rectified Flow Transformerscited by 93
  4. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 75
  5. 2025Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Conceptscited by 61
  6. 2024ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruningcited by 51
  7. 2024Direct Unlearning Optimization for Robust and Safe Text-to-Image Modelscited by 50
  8. 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 50
  9. 2024Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservationcited by 48
  10. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 47
  11. 2025GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modelingcited by 43
  12. 2025OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrievalcited by 40
38 more
  1. 2024STEREO: A Two-Stage Framework for Adversarially Robust Concept Erasing from Text-to-Image Diffusion Modelscited by 39
  2. 2024Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Conceptscited by 39
  3. 2024Precise, Fast, and Low-cost Concept Erasure in Value Space: Orthogonal Complement Matterscited by 38
  4. 20243DIS: Depth-Driven Decoupled Instance Synthesis for Text-to-Image Generationcited by 36
  5. 2025ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLMcited by 36
  6. 2024MIGC++: Advanced Multi-Instance Generation Controller for Image Synthesiscited by 35
  7. 2024Low-Rank Adaptation for Foundation Models: A Comprehensive Reviewcited by 35
  8. 2025Fantastic Targets for Concept Erasure in Diffusion Models and Where To Find Themcited by 34
  9. 2025Enhancing Time Series Forecasting through Selective Representation Spaces: A Patch Perspectivecited by 34
  10. 2025SPEED: Scalable, Precise, and Efficient Concept Erasure for Diffusion Modelscited by 33
  11. 2024ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Userscited by 32
  12. 2025HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrievalcited by 32
  13. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  14. 2025Towards Universal Modal Tracking with Online Dense Temporal Token Learningcited by 30
  15. 2025Does FLUX Already Know How to Perform Physically Plausible Image Composition?cited by 30
  16. 2024Erasing Concepts from Text-to-Image Diffusion Models with Few-shot Unlearningcited by 29
  17. 2025DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag Editingcited by 29
  18. 2025DreamRenderer: Taming Multi-Instance Attribute Control in Large-Scale Text-to-Image Modelscited by 28
  19. 2025Aurora: Towards Universal Generative Multimodal Time Series Forecastingcited by 28
  20. 2025Localized Concept Erasure for Text-to-Image Diffusion Models Using Training-Free Gated Low-Rank Adaptationcited by 27
  21. 2024TraSCE: Trajectory Steering for Concept Erasurecited by 25
  22. 2025CURE: Concept Unlearning via Orthogonal Representation Editing in Diffusion Modelscited by 24
  23. 2024Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shiftcited by 23
  24. 2024Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perceptioncited by 23
  25. 2025Exploring Decoupled Spatio-Temporal Consistency Learning and Self-Prompting Evolution for Self-Supervised Trackingcited by 23
  26. 2025From Zero to Detail: Deconstructing Ultra-High-Definition Image Restoration from Progressive Spectral Perspectivecited by 22
  27. 2024KinMo: Kinematic-aware Human Motion Understanding and Generationcited by 21
  28. 2025Distilling Diversity and Control in Diffusion Modelscited by 21
  29. 2025Self-Evolving Multi-Agent Systems via Textual Backpropagationcited by 21
  30. 2024Holistic Unlearning Benchmark: A Multi-Faceted Evaluation for Text-to-Image Diffusion Model Unlearningcited by 19
  31. 2025Training-Free Safe Denoisers for Safe Use of Diffusion Modelscited by 19
  32. 2024AlignGuard: Scalable Safety Alignment for Text-to-Image Generationcited by 17
  33. 2024Targeted Unlearning with Single Layer Unlearning Gradientcited by 16
  34. 2025Concept Pinpoint Eraser for Text-to-image Diffusion Models via Residual Attention Gatecited by 16
  35. 2025UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Datasetcited by 16
  36. 2025T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Modelcited by 16
  37. 2024RealEra: Semantic-level Concept Erasure via Neighbor-Concept Miningcited by 15
  38. 2025Sculpting Memory: Multi-Concept Forgetting in Diffusion Models via Dynamic Mask and Concept-Aware Optimizationcited by 15

50 most cited of 298 All 298 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Erasing Thousands of Concepts: Towards Scalable and Practical Concept Erasure for Text-to-Image Diffusion Models
  2. 2024Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models
  3. 2024Separable Multi-Concept Erasure from Diffusion Models
  4. 2023All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models
  5. 2026Compositional Concept Erasure in Text-to-Image Diffusion Models via Hierarchically Grounded Semantic Surgery
  6. 2025Fine-Grained Erasure in Text-to-Image Diffusion-based Foundation Models
  7. 2023Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers
  8. 2025When Are Concepts Erased From Diffusion Models?