# SPEED: Scalable, Precise, and Efficient Concept Erasure for Diffusion Models

arXiv: https://arxiv.org/abs/2503.07392 · HTML: https://mlsanity.com/paper/2503.07392/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 65 of its 88 references that are in the map.
- First posted on arXiv 10 Mar 2025.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 30 of 65)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Erasing Concepts from Diffusion Models](https://mlsanity.com/paper/2303.07345/) (2023)

## Built on by (most cited first, 30 of 36)

- [On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification](https://mlsanity.com/paper/2508.05629/) (2025)
- [Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Concepts](https://mlsanity.com/paper/2504.12782/) (2025)
- [Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?](https://mlsanity.com/paper/2509.03516/) (2025)
- [T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Model](https://mlsanity.com/paper/2510.22300/) (2025)
- [TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewards](https://mlsanity.com/paper/2512.07761/) (2025)
- [Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppression](https://mlsanity.com/paper/2505.19398/) (2025)
- [EraseFlow: Learning Concept Erasure Policies via GFlowNet-Driven Alignment](https://mlsanity.com/paper/2511.00804/) (2025)
- [Forget Many, Forget Right: Scalable and Precise Concept Unlearning in Diffusion Models](https://mlsanity.com/paper/2601.06162/) (2026)
- [MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Models](https://mlsanity.com/paper/2506.19257/) (2025)
- [Concept Unlearning by Modeling Key Steps of Diffusion Process](https://mlsanity.com/paper/2507.06526/) (2025)
- [Forget-It-All: Multi-Concept Machine Unlearning via Concept-Aware Neuron Masking](https://mlsanity.com/paper/2601.06163/) (2026)
- [Thinking with Frames: Generative Video Distortion Evaluation via Frame Reward Model](https://mlsanity.com/paper/2601.04033/) (2026)
- [Localized Concept Erasure in Text-to-Image Diffusion Models via High-Level Representation Misdirection](https://mlsanity.com/paper/2602.19631/) (2026)
- [To Erase, or Not to Erase: Robust Training-Free Concept Erasure with Preservation aware Adaptive Ranked Subspace Expansion](https://mlsanity.com/paper/2607.23492/) (2026)
- [VCE: Safe Autoregressive Image Generation via Visual Contrast Exploitation](https://mlsanity.com/paper/2509.16986/) (2025)
- [A Unified Framework for Diffusion Model Unlearning with f-Divergence](https://mlsanity.com/paper/2509.21167/) (2025)
- [GrOCE:Graph-Guided Online Concept Erasure for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2511.12968/) (2025)
- [LURE: Latent Space Unblocking for Multi-Concept Reawakening in Diffusion Models](https://mlsanity.com/paper/2601.14330/) (2026)
- [Orthogonal Concept Erasure for Diffusion Models](https://mlsanity.com/paper/2605.28902/) (2026)
- [Beyond Fixed Anchors: Precisely Erasing Concepts with Sibling Exclusive Counterparts](https://mlsanity.com/paper/2510.16342/) (2025)
- [EGLOCE: Training-Free Energy-Guided Latent Optimization for Concept Erasure](https://mlsanity.com/paper/2604.09405/) (2026)
- [Erasing Thousands of Concepts: Towards Scalable and Practical Concept Erasure for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2604.16481/) (2026)
- [What Concepts Lie Within? Detecting and Suppressing Risky Content in Diffusion Transformers](https://mlsanity.com/paper/2605.10180/) (2026)
- [Empty SPACE: Cross-Attention Sparsity for Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2605.10198/) (2026)
- [PEAK: Precise and Persistent Concept Erasure via k-Sparse Autoencoders](https://mlsanity.com/paper/2608.10985/) (2026)
- [RASteer: Retain-Aware Activation Steering for Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2610.01969/) (2026)
- [Your Unlearning Gives You Away: Identifying Erased Concepts in Diffusion Models](https://mlsanity.com/paper/2610.05601/) (2026)
- [Zero-Residual Concept Erasure via Progressive Alignment in Text-to-Image Model](https://mlsanity.com/paper/2508.04472/) (2025)
- [DICE: Disentangling Artist Style from Content via Contrastive Subspace Decomposition in Diffusion Models](https://mlsanity.com/paper/2602.08059/) (2026)
- [When Safe Concepts Become Unsafe: Multi-Concept Compositional Vulnerabilities in Text-to-Image Models](https://mlsanity.com/paper/2604.15967/) (2026)

## Abstract (arXiv)

Erasing concepts from large-scale text-to-image (T2I) diffusion models has become increasingly crucial due to the growing concerns over copyright infringement, offensive content, and privacy violations. In scalable applications, fine-tuning-based methods are time-consuming to precisely erase multiple target concepts, while real-time editing-based methods often degrade the generation quality of non-target concepts due to conflicting optimization objectives. To address this dilemma, we introduce SPEED, an efficient concept erasure approach that directly edits model parameters. SPEED searches for a null space, a model editing space where parameter updates do not affect non-target concepts, to achieve scalable and precise erasure. To facilitate accurate null space optimization, we incorporate three complementary strategies: Influence-based Prior Filtering (IPF) to selectively retain the most affected non-target concepts, Directed Prior Augmentation (DPA) to enrich the filtered retain set with semantically consistent variations, and Invariant Equality Constraints (IEC) to preserve key invariants during the T2I generation process. Extensive evaluations across multiple concept erasure tasks demonstrate that SPEED consistently outperforms existing methods in non-target preservation while achieving efficient and high-fidelity concept erasure, successfully erasing 100 concepts within only 5 seconds. Our code and models are available at: https://github.com/Ouxiang-Li/SPEED.