mlsanityData to 7 Oct 2026

DeltaGrad: Rapid retraining of machine learning models

arXiv 2006.14755 ↗First posted 26 Jun 2020cs.LG stat.MLcited by 143

Not on the map. Why?

Cited by 98 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Machine learning models are not static and may need to be retrained on slightly changed datasets, for instance, with the addition or deletion of a set of data points. This has many applications, including privacy, robustness, bias reduction, and uncertainty quantifcation. However, it is expensive to retrain models from scratch. To address this problem, we propose the DeltaGrad algorithm for rapid retraining machine learning models based on information cached during the training phase. We provide both theoretical and empirical support for the effectiveness of DeltaGrad, and show that it compares favorably to the state of the art.

Key facts. Cited by 143 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 20; 2023: 33; 2024: 40; 2025: 25; 2026: 12.

143 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  3. 2022A Survey of Machine Unlearningcited by 298
  4. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearningcited by 274
  5. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 251
  6. 2023In-Context Unlearning: Language Models as Few Shot Unlearnerscited by 214
  7. 2021Fast Yet Effective Machine Unlearningcited by 209
  8. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 175
  9. 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearningcited by 164
  10. 2023Boundary Unlearningcited by 132
  11. 2021Graph Unlearningcited by 129
  12. 2022Zero-Shot Machine Unlearningcited by 125
38 more
  1. 2022Continual Learning and Private Unlearningcited by 124
  2. 2022The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retrainingcited by 109
  3. 2021Federated Unlearning via Class-Discriminative Pruningcited by 108
  4. 2022Federated Unlearning: How to Efficiently Erase a Client in FL?cited by 90
  5. 2023Machine Unlearning: Solutions and Challengescited by 85
  6. 2022Towards Adversarial Evaluations for Inexact Machine Unlearningcited by 82
  7. 2024Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacycited by 78
  8. 2023Fast Model Debias with Machine Unlearningcited by 70
  9. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  10. 2022FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Informationcited by 53
  11. 2024Scissorhands: Scrub Data Influence via Connection Sensitivity in Networkscited by 50
  12. 2024Are we making progress in unlearning? Findings from the first NeurIPS unlearning competitioncited by 48
  13. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 44
  14. 2023GNNDelete: A General Strategy for Unlearning in Graph Neural Networkscited by 43
  15. 2021Certifiable Machine Unlearning for Linear Modelscited by 42
  16. 2022Multiplayer Performative Prediction: Learning in Decision-Dependent Gamescited by 42
  17. 2024Machine Unlearning Fails to Remove Data Poisoning Attackscited by 35
  18. 2024Machine Unlearning: A Comprehensive Surveycited by 33
  19. 2023Inductive Graph Unlearningcited by 30
  20. 2024A Review on Machine Unlearningcited by 30
  21. 2023Certified Minimax Unlearning with Generalization Rates and Deletion Capacitycited by 29
  22. 2021SSSE: Efficiently Erasing Samples from Trained Machine Learning Modelscited by 28
  23. 2023SAFE: Machine Unlearning With Shard Graphscited by 28
  24. 2024Corrective Machine Unlearningcited by 26
  25. 2023Fair Machine Unlearning: Data Removal while Mitigating Disparitiescited by 25
  26. 2022Verifiable and Provably Secure Machine Unlearningcited by 24
  27. 2022On the Trade-Off between Actionable Explanations and the Right to be Forgottencited by 20
  28. 2023Efficiently Forgetting What You Have Learned in Graph Representation Learning via Projectioncited by 20
  29. 2023MultiDelete for Multimodal Machine Unlearningcited by 20
  30. 2021DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networkscited by 19
  31. 2024A Unified and General Framework for Continual Learningcited by 19
  32. 2024Learning to Unlearn for Robust Machine Unlearningcited by 19
  33. 2023Machine Unlearning: A Surveycited by 18
  34. 2024Is Retain Set All You Need in Machine Unlearning? Restoring Performance of Unlearned Models with Out-Of-Distribution Imagescited by 18
  35. 2024Attribute-to-Delete: Machine Unlearning via Datamodel Matchingcited by 18
  36. 2025LLM Unlearning with LLM Beliefscited by 17
  37. 2022Efficient Attribute Unlearning: Towards Selective Removal of Input Attributes from Feature Representationscited by 16
  38. 2024MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearningcited by 16

50 most cited of 143 All 143 (JSON)