mlsanityData to 14 Oct 2026

ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning

arXiv 2405.19237 ↗First posted 29 May 2024cs.CV cs.AI cs.LGcited by 55

On the map: Diffusion models → Robustness, privacy and adversarial aspects

also in 1 more
Abstract (arXiv)

While large-scale text-to-image diffusion models have demonstrated impressive image-generation capabilities, there are significant concerns about their potential misuse for generating unsafe content, violating copyright, and perpetuating societal biases. Recently, the text-to-image generation community has begun addressing these concerns by editing or unlearning undesired concepts from pre-trained models. However, these methods often involve data-intensive and inefficient fine-tuning or utilize various forms of token remapping, rendering them susceptible to adversarial jailbreaks. In this paper, we present a simple and effective training-free approach, ConceptPrune, wherein we first identify critical regions within pre-trained models responsible for generating undesirable concepts, thereby facilitating straightforward concept unlearning via weight pruning. Experiments across a range of concepts including artistic styles, nudity, object erasure, and gender debiasing demonstrate that target concepts can be efficiently erased by pruning a tiny fraction, approximately 0.12% of total weights, enabling multi-concept erasure and robustness against various white-box and black-box adversarial attacks.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 61 of its 70 references that are in the map. First posted on arXiv 29 May 2024.

Builds on

61 of 70 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models24 May
  2. 2024MACE: Mass Concept Erasure in Diffusion Models10 Mar
  3. 2024Separable Multi-Concept Erasure from Diffusion Models3 Feb
  4. 2024Erasing Undesirable Influence in Diffusion Models11 Jan
  5. 2023Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers29 Nov
  6. 2023MMA-Diffusion: MultiModal Attack on Diffusion Models29 Nov
  7. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation19 Oct
  8. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now18 Oct
  9. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?16 Oct
  10. 2023Implicit Concept Removal of Diffusion Models9 Oct
  11. 2023Unified Concept Editing in Diffusion Models25 Aug
  12. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models30 Mar
3 more
  1. 2023Ablating Concepts in Text-to-Image Diffusion Models23 Mar
  2. 2023Erasing Concepts from Diffusion Models13 Mar
  3. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models9 Nov

Other references · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2020Denoising Diffusion Implicit Modelscited by 9,781
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
34 more
  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  2. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  4. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  6. 2020GLU Variants Improve Transformercited by 1,804
  7. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  9. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  10. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  11. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  12. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  13. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
  14. 2018Rethinking the Value of Network Pruningcited by 599
  15. 2020What is the State of Neural Network Pruning?cited by 583
  16. 2014Understanding Deep Image Representations by Inverting Themcited by 403
  17. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 231
  18. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 226
  19. 2023Structural Pruning for Diffusion Modelscited by 195
  20. 2020fastai: A Layered API for Deep Learningcited by 183
  21. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 177
  22. 2021MoEfication: Transformer Feed-forward Layers are Mixtures of Expertscited by 152
  23. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 130
  24. 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
  25. 2020Analyzing Individual Neurons in Pre-trained Language Modelscited by 97
  26. 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 85
  27. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 66
  28. 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 62
  29. 2024Scissorhands: Scrub Data Influence via Connection Sensitivity in Networkscited by 52
  30. 2023The Journey, Not the Destination: How Data Guides Diffusion Modelscited by 43
  31. 2023Emergent Modularity in Pre-trained Transformerscited by 37
  32. 2020Finding Experts in Transformer Modelscited by 28
  33. 2023A Survey of Diffusion Based Image Generation Models: Issues and Their Solutionscited by 25
  34. 2022Discovering Salient Neurons in Deep NLP Modelscited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 22; 2026: 25.

55 papers.

Sort:

By area:

  1. 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 133
  2. 2025Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Conceptscited by 62
  3. 2024Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Surveycited by 35
  4. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 32
  5. 2024MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Modelcited by 26
  6. 2024AlignGuard: Scalable Safety Alignment for Text-to-Image Generationcited by 20
  7. 2025Sculpting Memory: Multi-Concept Forgetting in Diffusion Models via Dynamic Mask and Concept-Aware Optimizationcited by 19
  8. 2024Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Modelscited by 16
  9. 2026Mass Concept Erasure in Diffusion Models with Concept Hierarchycited by 14
  10. 2025Minimalist Concept Erasure in Generative Modelscited by 13
  11. 2024Unveiling Concept Attribution in Diffusion Modelscited by 11
  12. 2025Concept Corrector: Erase concepts on the fly for text-to-image diffusion modelscited by 11
38 more
  1. 2025Revoking Amnesia: RL-based Trajectory Optimization to Resurrect Erased Concepts in Diffusion Modelscited by 11
  2. 2025Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppressioncited by 10
  3. 2025Image Can Bring Your Memory Back: A Novel Multi-Modal Guided Attack against Image Generation Model Unlearningcited by 10
  4. 2025SAEmnesia: Erasing Concepts in Diffusion Models with Supervised Sparse Autoencoderscited by 10
  5. 2025Erased or Dormant? Rethinking Concept Erasure Through Reversibilitycited by 9
  6. 2026Forget Many, Forget Right: Scalable and Precise Concept Unlearning in Diffusion Modelscited by 9
  7. 2025BackdoorDM: A Comprehensive Benchmark for Backdoor Learning on Diffusion Modelcited by 8
  8. 2025Concept Steerers: Leveraging K-Sparse Autoencoders for Test-Time Controllable Generationscited by 7
  9. 2025TarPro: Targeted Protection against Malicious Image Editingcited by 7
  10. 2025ACE: Concept Editing in Diffusion Models without Performance Degradationcited by 6
  11. 2025PurifyGen: A Risk-Discrimination and Semantic-Purification Model for Safe Text-to-Image Generationcited by 5
  12. 2026Differential Vector Erasure: Unified Training-Free Concept Erasure for Flow Matching Modelscited by 5
  13. 2025Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocolscited by 4
  14. 2026ActErase: A Training-Free Paradigm for Precise Concept Erasure via Activation Redirectioncited by 4
  15. 2026Forget-It-All: Multi-Concept Machine Unlearning via Concept-Aware Neuron Maskingcited by 4
  16. 2025Erased, But Not Forgotten: Erased Rectified Flow Transformers Still Remain Unsafe Under Concept Attackcited by 3
  17. 2025Continual Personalization for Diffusion Modelscited by 3
  18. 2026SafeRedir: Prompt Embedding Redirection for Robust Unlearning in Image Generation Modelscited by 3
  19. 2026Closed-Form Concept Erasure via Double Projectionscited by 3
  20. 2025Demystifying Foreground-Background Memorization in Diffusion Modelscited by 2
  21. 2025VCE: Safe Autoregressive Image Generation via Visual Contrast Exploitationcited by 2
  22. 2025Value-Aligned Prompt Moderation via Zero-Shot Agentic Rewriting for Safe Image Generationcited by 2
  23. 2026FlowErase-RL: Rethinking Concept Erasure as Reward Optimization in Flow Matching Modelscited by 2
  24. 2024Red-Teaming Text-to-Image Models via In-Context Experience Replay and Semantic-Preserving Prompt Rewritingcited by 1
  25. 2025Personalized Safety Alignment for Text-to-Image Diffusion Modelscited by 1
  26. 2026Roots Beneath the Cut: Uncovering the Risk of Concept Revival in Pruning-Based Unlearning for Diffusion Modelscited by 1
  27. 2026Prototype-Guided Concept Erasure in Diffusion Modelscited by 1
  28. 2026PROBE: Diagnosing Residual Concept Capacity in Erased Text-to-Video Diffusion Modelscited by 1
  29. 2026Empty SPACE: Cross-Attention Sparsity for Concept Erasure in Diffusion Modelscited by 1
  30. 2026FlowErase-OPD: Multi-Concept Erasure via Anchored On-Policy Distillation in Flow Matching Modelscited by 1
  31. 2026FADE: Frame-Aware Diffusion-Transformer-based Multi-Concept Erasure for Video Unlearningcited by 1
  32. 2026No Concept Escapes the Audit: Auditing-Aware Unlearning for Verifiable Concept Erasure in Diffusion Modelscited by 1
  33. 2026Your Unlearning Gives You Away: Identifying Erased Concepts in Diffusion Modelscited by 1
  34. 2025SP-Guard: Selective Prompt-adaptive Guidance for Safe Text-to-Image Generationcited by 0
  35. 2026REFORGE: Multi-modal Attacks Reveal Vulnerable Concept Unlearning in Image Generation Modelscited by 0
  36. 2026CoreUnlearn: Rethinking Concept Unlearning through Disentangled Component-Level Erasure in Text-guided Diffusion Modelscited by 0
  37. 2026SACE: Concept Erasure at the Semantic Singularity in Visual Autoregressive Modelscited by 0
  38. 2026Concept Removal for Frontier Image Generative Modelscited by 0

50 most cited of 55 All 55 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Pruning for Robust Concept Erasing in Diffusion Models
  2. 2024Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradient
  3. 2025Robust Concept Erasure in Diffusion Models: A Theoretical Perspective on Security and Robustness
  4. 2025A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Models
  5. 2025ACE: Attentional Concept Erasure in Diffusion Models
  6. 2026Projected Gradient Unlearning for Text-to-Image Diffusion Models: Defending Against Concept Revival Attacks
  7. 2024Erasing Concepts from Text-to-Image Diffusion Models with Few-shot Unlearning
  8. 2026eval-unlearn: Benchmarking unlearning in Text-to-Image Diffusion Models