mlsanityData to 7 Oct 2026

Descent-to-Delete: Gradient-Based Methods for Machine Unlearning

arXiv 2007.02923 ↗First posted 6 Jul 2020stat.ML cs.LGcited by 303

Not on the map. Why?

Cited by 255 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We study the data deletion problem for convex models. By leveraging techniques from convex optimization and reservoir sampling, we give the first data deletion algorithms that are able to handle an arbitrarily long sequence of adversarial updates while promising both per-deletion run-time and steady-state error that do not grow with the length of the update sequence. We also introduce several new conceptual distinctions: for example, we can ask that after a deletion, the entire state maintained by the optimization algorithm is statistically indistinguishable from the state that would have resulted had we retrained, or we can ask for the weaker condition that only the observable output is statistically indistinguishable from the observable output that would have resulted from retraining. We are able to give more efficient deletion algorithms under this weaker deletion criterion.

Key facts. Cited by 303 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 30; 2023: 43; 2024: 79; 2025: 76; 2026: 61.

303 papers.

Sort:

By area:

  1. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 376
  2. 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 360
  3. 2022A Survey of Machine Unlearningcited by 298
  4. 2023Large Language Model Unlearningcited by 288
  5. 2023Towards Unbounded Machine Unlearningcited by 261
  6. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 251
  7. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  8. 2021Machine Unlearning of Features and Labelscited by 223
  9. 2023In-Context Unlearning: Language Models as Few Shot Unlearnerscited by 214
  10. 2021Fast Yet Effective Machine Unlearningcited by 209
  11. 2023Model Sparsity Can Simplify Machine Unlearningcited by 206
  12. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
38 more
  1. 2022Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teachercited by 189
  2. 2021Adaptive Machine Unlearningcited by 164
  3. 2021Graph Unlearningcited by 129
  4. 2021Machine Unlearning via Algorithmic Stabilitycited by 128
  5. 2022Zero-Shot Machine Unlearningcited by 125
  6. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  7. 2022The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retrainingcited by 109
  8. 2021Federated Unlearning via Class-Discriminative Pruningcited by 108
  9. 2022Federated Unlearning: How to Efficiently Erase a Client in FL?cited by 90
  10. 2023Machine Unlearning: Solutions and Challengescited by 85
  11. 2022Deep Unlearning via Randomized Conditionally Independent Hessianscited by 83
  12. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearningcited by 81
  13. 2024Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacycited by 78
  14. 2023DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Modelscited by 75
  15. 2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 72
  16. 2022Recommendation Unlearningcited by 71
  17. 2023Fast Model Debias with Machine Unlearningcited by 70
  18. 2022Backdoor Defense with Machine Unlearningcited by 61
  19. 2023GIF: A General Graph Unlearning Strategy via Influence Functioncited by 59
  20. 2022Forget Unlearning: Towards True Data-Deletion in Machine Learningcited by 58
  21. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progresscited by 58
  22. 2024Scissorhands: Scrub Data Influence via Connection Sensitivity in Networkscited by 50
  23. 2024Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearningcited by 48
  24. 2024Are we making progress in unlearning? Findings from the first NeurIPS unlearning competitioncited by 48
  25. 2021Hard to Forget: Poisoning Attacks on Certified Machine Unlearningcited by 47
  26. 2024Machine Unlearning for Image-to-Image Generative Modelscited by 47
  27. 2024UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Modelscited by 47
  28. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  29. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearningcited by 44
  30. 2023GNNDelete: A General Strategy for Unlearning in Graph Neural Networkscited by 43
  31. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancementcited by 43
  32. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Stepscited by 43
  33. 2021Certifiable Machine Unlearning for Linear Modelscited by 42
  34. 2022Evaluating Machine Unlearning via Epistemic Uncertaintycited by 39
  35. 2022Algorithms that Approximate Data Removal: New Results and Limitationscited by 39
  36. 2025When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformerscited by 38
  37. 2021Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networkscited by 37
  38. 2022Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attackscited by 37

50 most cited of 303 All 303 (JSON)