mlsanityData to 7 Oct 2026

Machine Unlearning via Algorithmic Stability

arXiv 2102.13179 ↗First posted 25 Feb 2021cs.LG stat.MLcited by 128

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

We study the problem of machine unlearning and identify a notion of algorithmic stability, Total Variation (TV) stability, which we argue, is suitable for the goal of exact unlearning. For convex risk minimization problems, we design TV-stable algorithms based on noisy Stochastic Gradient Descent (SGD). Our key contribution is the design of corresponding efficient unlearning algorithms, which are based on constructing a (maximal) coupling of Markov chains for the noisy SGD procedure. To understand the trade-offs between accuracy and unlearning efficiency, we give upper and lower bounds on excess empirical and populations risk of TV stable algorithms for convex risk minimization. Our techniques generalize to arbitrary non-convex functions, and our algorithms are differentially private as well.

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 13 of its 30 references that are in the map. First posted on arXiv 25 Feb 2021.

Builds on

13 of 30 references.

References · 13

  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2017Renyi Differential Privacycited by 643
  3. 2019Certified Data Removal from Machine Learning Modelscited by 512
  4. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  5. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  6. 2020Approximate Data Deletion from Machine Learning Modelscited by 250
  7. 2018Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergencescited by 237
  8. 2019Private Stochastic Convex Optimization with Optimal Ratescited by 183
  9. 2016Katyusha: The First Direct Acceleration of Stochastic Gradient Methodscited by 134
  10. 2014Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Boundscited by 104
  11. 2015Algorithmic Stability for Adaptive Data Analysiscited by 88
  12. 2018How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGDcited by 55
1 more
  1. 2016Tight Complexity Bounds for Optimizing Composite Objectivescited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 12; 2023: 24; 2024: 37; 2025: 36; 2026: 17.

128 papers.

Sort:

By area:

  1. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 376
  2. 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 360
  3. 2022A Survey of Machine Unlearningcited by 298
  4. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 251
  5. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  6. 2023Model Sparsity Can Simplify Machine Unlearningcited by 206
  7. 2022Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teachercited by 189
  8. 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearningcited by 164
  9. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  10. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearningcited by 104
  11. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyondcited by 76
  12. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
38 more
  1. 2023GIF: A General Graph Unlearning Strategy via Influence Functioncited by 59
  2. 2022Forget Unlearning: Towards True Data-Deletion in Machine Learningcited by 58
  3. 2024Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearningcited by 48
  4. 2024UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Modelscited by 47
  5. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 44
  6. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearningcited by 44
  7. 2023GNNDelete: A General Strategy for Unlearning in Graph Neural Networkscited by 43
  8. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancementcited by 43
  9. 2022Evaluating Machine Unlearning via Epistemic Uncertaintycited by 39
  10. 2022Algorithms that Approximate Data Removal: New Results and Limitationscited by 39
  11. 2022Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attackscited by 37
  12. 2022Deep Regression Unlearningcited by 32
  13. 2023Inductive Graph Unlearningcited by 30
  14. 2024A Review on Machine Unlearningcited by 30
  15. 2023Certified Minimax Unlearning with Generalization Rates and Deletion Capacitycited by 29
  16. 2024Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning ratecited by 29
  17. 2023SAFE: Machine Unlearning With Shard Graphscited by 28
  18. 2024Towards Certified Unlearning for Deep Neural Networkscited by 28
  19. 2023Towards Machine Unlearning Benchmarks: Forgetting the Personal Identities in Facial Recognition Systemscited by 27
  20. 2024WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Modelscited by 25
  21. 2024A Probabilistic Perspective on Unlearning and Alignment for Large Language Modelscited by 21
  22. 2022Knowledge Removal in Sampling-based Bayesian Inferencecited by 20
  23. 2023Efficiently Forgetting What You Have Learned in Graph Representation Learning via Projectioncited by 20
  24. 2023MultiDelete for Multimodal Machine Unlearningcited by 20
  25. 2024Verification of Machine Unlearning is Fragilecited by 20
  26. 2024A Unified and General Framework for Continual Learningcited by 19
  27. 2024CLIPErase: Efficient Unlearning of Visual-Textual Associations in CLIPcited by 19
  28. 2022Data Redaction from Pre-trained GANscited by 18
  29. 2023Machine Unlearning: A Surveycited by 18
  30. 2023Tight Bounds for Machine Unlearning via Differential Privacycited by 18
  31. 2023Statistical Indistinguishability of Learning Algorithmscited by 17
  32. 2024Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functionscited by 17
  33. 2025Tool Unlearning for Tool-Augmented LLMscited by 17
  34. 2023Training Data Protection with Compositional Diffusion Modelscited by 16
  35. 2024Communication Efficient and Provable Federated Unlearningcited by 16
  36. 2024MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearningcited by 16
  37. 2024MEOW: MEMOry Supervised LLM Unlearning Via Inverted Factscited by 16
  38. 2023ERASER: Machine Unlearning in MLaaS via an Inference Serving-Aware Approachcited by 15

50 most cited of 128 All 128 (JSON)