mlsanityData to 7 Oct 2026

Model Sparsity Can Simplify Machine Unlearning

arXiv 2304.04934 ↗First posted 11 Apr 2023cs.LGcited by 206

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

In response to recent data regulation requirements, machine unlearning (MU) has emerged as a critical process to remove the influence of specific examples from a given model. Although exact unlearning can be achieved through complete model retraining using the remaining dataset, the associated computational costs have driven the development of efficient, approximate unlearning techniques. Moving beyond data-centric MU approaches, our study introduces a novel model-based perspective: model sparsification via weight pruning, which is capable of reducing the gap between exact unlearning and approximate unlearning. We show in both theory and practice that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner, closing the approximation gap, while continuing to be efficient. This leads to a new MU paradigm, termed prune first, then unlearn, which infuses a sparse model prior into the unlearning process. Building on this insight, we also develop a sparsity-aware unlearning method that utilizes sparsity regularization to enhance the training process of approximate unlearning. Extensive experiments show that our proposals consistently benefit MU in various unlearning scenarios. A notable highlight is the 77% unlearning efficacy gain of fine-tuning (one of the simplest unlearning methods) when using sparsity-aware unlearning. Furthermore, we demonstrate the practical impact of our proposed MU methods in addressing other machine learning challenges, such as defending against backdoor attacks and enhancing transfer learning. Codes are available at https://github.com/OPTML-Group/Unlearn-Sparse.

Key facts. Cited by 206 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 61 of its 80 references that are in the map. First posted on arXiv 11 Apr 2023.

Builds on

61 of 80 references.

Closest ancestors · 8

Same direction, newest first

  1. 2022A Survey of Machine Unlearning6 Sep
  2. 2022Evaluating Machine Unlearning via Epistemic Uncertainty23 Aug
  3. 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning22 Oct
  4. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep
  5. 2021Hard to Forget: Poisoning Attacks on Certified Machine Unlearning17 Sep
  6. 2021Machine Unlearning of Features and Labels26 Aug
  7. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar
  8. 2021Machine Unlearning via Algorithmic Stability25 Feb

Other references · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  4. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  5. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  6. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  7. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  8. 2019Machine Unlearningcited by 1,020
  9. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  10. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 627
  11. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  12. 2018Rethinking the Value of Network Pruningcited by 595
38 more
  1. 2020What is the State of Neural Network Pruning?cited by 578
  2. 2023Erasing Concepts from Diffusion Modelscited by 530
  3. 2019Certified Data Removal from Machine Learning Modelscited by 512
  4. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  5. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  6. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  7. 2020Amnesiac Machine Learningcited by 305
  8. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  9. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 297
  10. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 273
  11. 2020Approximate Data Deletion from Machine Learning Modelscited by 250
  12. 2020Coresets via Bilevel Optimization for Continual Learning and Streamingcited by 191
  13. 2020Pruning Neural Networks at Initialization: Why are We Missing the Mark?cited by 181
  14. 2019What Do Compressed Deep Neural Networks Forget?cited by 164
  15. 2019Privacy Risks of Securing Machine Learning Models against Adversarial Examplescited by 136
  16. 2022Dataset Pruning: Reducing Training Data by Examining Generalization Influencecited by 132
  17. 2021Graph Unlearningcited by 129
  18. 2022The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retrainingcited by 109
  19. 2016On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimizationcited by 108
  20. 2021Federated Unlearning via Class-Discriminative Pruningcited by 108
  21. 2020The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Modelscited by 91
  22. 2021Leveraging Sparse Linear Layers for Debuggable Deep Networkscited by 82
  23. 2021A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustnesscited by 62
  24. 2022Backdoor Defense with Machine Unlearningcited by 61
  25. 2022Advancing Model Pruning via Bi-level Optimizationcited by 55
  26. 2022FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Informationcited by 53
  27. 2021Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?cited by 52
  28. 2022Certified Graph Unlearningcited by 46
  29. 2023GNNDelete: A General Strategy for Unlearning in Graph Neural Networkscited by 43
  30. 2022Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attackscited by 37
  31. 2020Against Membership Inference Attack: Pruning is All You Needcited by 32
  32. 2022A Data-Based Perspective on Transfer Learningcited by 32
  33. 2022Learning with Recoverable Forgettingcited by 29
  34. 2022Understanding Instance-Level Impact of Fairness Constraintscited by 27
  35. 2022Fair Infinitesimal Jackknife: Mitigating the Influence of Biased Training Data Points Without Refittingcited by 26
  36. 2022Can Model Compression Improve NLP Fairnesscited by 18
  37. 2022Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Freecited by 15
  38. 2020A Privacy-Preserving-Oriented DNN Pruning and Mobile Acceleration Frameworkcited by 13

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 65; 2025: 83; 2026: 53.

206 papers.

Sort:

By area:

  1. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  2. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 204
  3. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 112
  4. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearningcited by 104
  5. 2024What makes unlearning hard and what to do about itcited by 89
  6. 2024Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacycited by 78
  7. 2024LLM Unlearning via Loss Adjustment with Only Forget Datacited by 71
  8. 2024Are we making progress in unlearning? Findings from the first NeurIPS unlearning competitioncited by 48
  9. 2024UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Modelscited by 47
  10. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 44
  11. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearningcited by 44
  12. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancementcited by 43
38 more
  1. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearningcited by 39
  2. 2024CLEAR: Character Unlearning in Textual and Visual Modalitiescited by 36
  3. 2024Threats, Attacks, and Defenses in Machine Unlearning: A Surveycited by 34
  4. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspectivecited by 33
  5. 2024Erasing Concepts from Text-to-Image Diffusion Models with Few-shot Unlearningcited by 29
  6. 2024Towards Certified Unlearning for Deep Neural Networkscited by 28
  7. 2024Corrective Machine Unlearningcited by 26
  8. 2024WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Modelscited by 25
  9. 2024Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Modelscited by 24
  10. 2024Pruning for Robust Concept Erasing in Diffusion Modelscited by 22
  11. 2024Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Modelscited by 22
  12. 2024Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillationcited by 21
  13. 2023MultiDelete for Multimodal Machine Unlearningcited by 20
  14. 2024Towards Robust Evaluation of Unlearning in LLMs via Data Transformationscited by 19
  15. 2025Learning to Unlearn while Retaining: Combating Gradient Conflicts in Machine Unlearningcited by 19
  16. 2024Is Retain Set All You Need in Machine Unlearning? Restoring Performance of Unlearned Models with Out-Of-Distribution Imagescited by 18
  17. 2024Attribute-to-Delete: Machine Unlearning via Datamodel Matchingcited by 18
  18. 2025Tool Unlearning for Tool-Augmented LLMscited by 17
  19. 2025GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detectioncited by 17
  20. 2024MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearningcited by 16
  21. 2024Targeted Unlearning with Single Layer Unlearning Gradientcited by 16
  22. 2025Are We Truly Forgetting? A Critical Re-examination of Machine Unlearning Evaluation Protocolscited by 16
  23. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  24. 2025NoT: Federated Unlearning via Weight Negationcited by 15
  25. 2024Contrastive Unlearning: A Contrastive Approach to Machine Unlearningcited by 14
  26. 2024Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysiscited by 14
  27. 2025Exact Unlearning of Finetuning Data via Model Merging at Scalecited by 14
  28. 2025RULE: Reinforcement UnLEarning Achieves Forget-Retain Pareto Optimalitycited by 14
  29. 2023DUCK: Distance-based Unlearning via Centroid Kinematicscited by 13
  30. 2024Remaining-data-free Machine Unlearning by Suppressing Sample Contributioncited by 13
  31. 2024Probing Unlearned Diffusion Models: A Transferable Adversarial Attack Perspectivecited by 13
  32. 2024Decoupling the Class Label and the Target Concept in Machine Unlearningcited by 12
  33. 2024Mitigating Selection Bias with Node Pruning and Auxiliary Optionscited by 12
  34. 2024Human-inspired Perspectives: A Survey on AI Long-term Memorycited by 12
  35. 2025Robust Machine Unlearning for Quantized Neural Networks via Adaptive Gradient Reweighting with Similar Labelscited by 12
  36. 2025From Dormant to Deleted: Tamper-Resistant Unlearning Through Weight-Space Regularizationcited by 12
  37. 2025Ascent Fails to Forgetcited by 12
  38. 2024Towards Natural Machine Unlearningcited by 11

50 most cited of 206 All 206 (JSON)