mlsanityData to 7 Oct 2026

Remember What You Want to Forget: Algorithms for Machine Unlearning

arXiv 2103.03279 ↗First posted 4 Mar 2021cs.LG cs.AIcited by 360

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

We study the problem of unlearning datapoints from a learnt model. The learner first receives a dataset S drawn i.i.d. from an unknown distribution, and outputs a model widehatw that performs well on unseen samples from the same distribution. However, at some point in the future, any training datapoint z ∈ S can request to be unlearned, thus prompting the learner to modify its output model while still ensuring the same accuracy guarantees. We initiate a rigorous study of generalization in machine unlearning, where the goal is to perform well on previously unseen datapoints. Our focus is on both computational and storage complexity. For the setting of convex losses, we provide an unlearning algorithm that can unlearn up to O(n/d1/4) samples, where d is the problem dimension. In comparison, in general, differentially private learning (which implies unlearning) only guarantees deletion of O(n/d1/2) samples. This demonstrates a novel separation between differential privacy and machine unlearning.

Key facts. Cited by 360 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 19 of its 39 references that are in the map. First posted on arXiv 4 Mar 2021.

Builds on

19 of 39 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Machine Unlearning via Algorithmic Stability25 Feb

Other references · 18

  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  2. 2019Machine Unlearningcited by 1,020
  3. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  4. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  5. 2014Convex Optimization: Algorithms and Complexitycited by 569
  6. 2019Certified Data Removal from Machine Learning Modelscited by 512
  7. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  8. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  9. 2020Approximate Data Deletion from Machine Learning Modelscited by 250
  10. 2016Robust Estimators in High Dimensions without the Computational Intractabilitycited by 209
  11. 2019Private Stochastic Convex Optimization with Optimal Ratescited by 183
  12. 2020When Machine Unlearning Jeopardizes Privacycited by 181
6 more
  1. 2020Mixed-Privacy Forgetting in Deep Networkscited by 158
  2. 2016Agnostic Estimation of Mean and Covariancecited by 131
  3. 2020Variational Bayesian Unlearningcited by 114
  4. 2020Formalizing Data Deletion in the Context of the Right to be Forgottencited by 46
  5. 2020Towards Probabilistic Verification of Machine Unlearningcited by 46
  6. 2016Generalization of ERM in Stochastic Convex Optimization: The Dimension Strikes Backcited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 26; 2023: 54; 2024: 99; 2025: 92; 2026: 81.

360 papers.

Sort:

By area:

  1. 2023Erasing Concepts from Diffusion Modelscited by 530
  2. 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 507
  3. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
  4. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 376
  5. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  6. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
  7. 2022A Survey of Machine Unlearningcited by 298
  8. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearningcited by 274
  9. 2023Towards Unbounded Machine Unlearningcited by 261
  10. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 251
  11. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  12. 2023In-Context Unlearning: Language Models as Few Shot Unlearnerscited by 214
38 more
  1. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMscited by 211
  2. 2021Fast Yet Effective Machine Unlearningcited by 209
  3. 2023Model Sparsity Can Simplify Machine Unlearningcited by 206
  4. 2022Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teachercited by 189
  5. 2023Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampeningcited by 177
  6. 2021Adaptive Machine Unlearningcited by 164
  7. 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearningcited by 164
  8. 2024Towards Safer Large Language Models through Machine Unlearningcited by 134
  9. 2023Boundary Unlearningcited by 132
  10. 2022Zero-Shot Machine Unlearningcited by 125
  11. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  12. 2024Machine Unlearning of Pre-trained Large Language Modelscited by 118
  13. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  14. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearningcited by 104
  15. 2022Federated Unlearning: How to Efficiently Erase a Client in FL?cited by 90
  16. 2023Machine Unlearning: Solutions and Challengescited by 85
  17. 2024Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Differencecited by 85
  18. 2022Deep Unlearning via Randomized Conditionally Independent Hessianscited by 83
  19. 2024Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacycited by 78
  20. 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 76
  21. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  22. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  23. 2022Forget Unlearning: Towards True Data-Deletion in Machine Learningcited by 58
  24. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progresscited by 58
  25. 2024Erasing Undesirable Influence in Diffusion Modelscited by 52
  26. 2024Scissorhands: Scrub Data Influence via Connection Sensitivity in Networkscited by 50
  27. 2024Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearningcited by 48
  28. 2024Are we making progress in unlearning? Findings from the first NeurIPS unlearning competitioncited by 48
  29. 2021Hard to Forget: Poisoning Attacks on Certified Machine Unlearningcited by 47
  30. 2024UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Modelscited by 47
  31. 2022Certified Graph Unlearningcited by 46
  32. 2024UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AIcited by 46
  33. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 44
  34. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearningcited by 44
  35. 2023GNNDelete: A General Strategy for Unlearning in Graph Neural Networkscited by 43
  36. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancementcited by 43
  37. 2024Soft Prompting for Unlearning in Large Language Modelscited by 42
  38. 2022Evaluating Machine Unlearning via Epistemic Uncertaintycited by 39

50 most cited of 360 All 360 (JSON)