mlsanityData to 7 Oct 2026

Exploring Incremental Unlearning: Techniques, Challenges, and Future Directions

arXiv 2502.16708 ↗First posted 23 Feb 2025cs.LG cs.AIcited by 2

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

The growing demand for data privacy in Machine Learning (ML) applications has seen Machine Unlearning (MU) emerge as a critical area of research. As the `right to be forgotten' becomes regulated globally, it is increasingly important to develop mechanisms that delete user data from AI systems while maintaining performance and scalability of these systems. Incremental Unlearning (IU) is a promising MU solution to address the challenges of efficiently removing specific data from ML models without the need for expensive and time-consuming full retraining. This paper presents the various techniques and approaches to IU. It explores the challenges faced in designing and implementing IU mechanisms. Datasets and metrics for evaluating the performance of unlearning techniques are discussed as well. Finally, potential solutions to the IU challenges alongside future research directions are offered. This survey provides valuable insights for researchers and practitioners seeking to understand the current landscape of IU and its potential for enhancing privacy-preserving intelligent systems.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 103 of its 157 references that are in the map. First posted on arXiv 23 Feb 2025.

Builds on

103 of 157 references.

Closest ancestors · 45

Same direction, newest first

  1. 2024Verifying Machine Unlearning with Explainable AI20 Nov
  2. 2024Neural Corrective Machine Unranking13 Nov
  3. 2024Edge Unlearning is Not "on Edge"! An Adaptive Exact Unlearning System on Resource-Constrained Devices14 Oct
  4. 2024Survey of Security and Data Attacks on Machine Unlearning In Financial and E-Commerce29 Sep
  5. 2024Parallel Unlearning in Inherited Model Networks16 Aug
  6. 2024On the Limitations and Prospects of Machine Unlearning for Generative AI1 Aug
  7. 2024Machine Unlearning in Generative AI: A Survey30 Jul
  8. 2024Learning to Unlearn for Robust Machine Unlearning15 Jul
  9. 2024On Large Language Model Continual Unlearning14 Jul
  10. 2024Digital Forgetting in Large Language Models: A Survey of Unlearning Methods2 Apr
  11. 2024The Frontier of Data Erasure: Machine Unlearning for Large Language Models23 Mar
  12. 2024Threats, Attacks, and Defenses in Machine Unlearning: A Survey20 Mar
33 more
  1. 2024Machine Unlearning: Taxonomy, Metrics, Applications, Challenges, and Prospects13 Mar
  2. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning12 Mar
  3. 2024Dissecting Language Models: Machine Unlearning via Selective Pruning2 Mar
  4. 2024Loss-Free Machine Unlearning29 Feb
  5. 2024Corrective Machine Unlearning21 Feb
  6. 2024Towards Safer Large Language Models through Machine Unlearning15 Feb
  7. 2024Selective Forgetting: Advancing Machine Unlearning Techniques and Evaluation in Language Models8 Feb
  8. 2024Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampening6 Feb
  9. 2024Machine Unlearning in Large Language Models3 Feb
  10. 2024eCIL-MU: Embedding based Class Incremental Learning and Machine Unlearning4 Jan
  11. 2023Layer Attack Unlearning: Fast and Accurate Machine Unlearning via Layer Level Attack and Knowledge Distillation28 Dec
  12. 2023Reinforcement Unlearning26 Dec
  13. 2023Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection7 Dec
  14. 2023Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges27 Nov
  15. 2023ERASER: Machine Unlearning in MLaaS via an Inference Serving-Aware Approach3 Nov
  16. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMs31 Oct
  17. 2023Fast Model Debias with Machine Unlearning19 Oct
  18. 2023Large Language Model Unlearning14 Oct
  19. 2023A Duty to Forget, a Right to be Assured? Exposing Vulnerabilities in Machine Unlearning Services15 Sep
  20. 2023Machine Unlearning: Solutions and Challenges14 Aug
  21. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy10 May
  22. 2023Model Sparsity Can Simplify Machine Unlearning11 Apr
  23. 2023Towards Unbounded Machine Unlearning20 Feb
  24. 2023Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers27 Jan
  25. 2022Proof of Unlearning: Definitions and Instantiation20 Oct
  26. 2022Deep Regression Unlearning15 Oct
  27. 2022A Survey of Machine Unlearning6 Sep
  28. 2022Zero-Shot Machine Unlearning14 Jan
  29. 2021Fast Yet Effective Machine Unlearning17 Nov
  30. 2021Hard to Forget: Poisoning Attacks on Certified Machine Unlearning17 Sep
  31. 2021Machine Unlearning of Features and Labels26 Aug
  32. 2021Adaptive Machine Unlearning8 Jun
  33. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar

Other references · 58

  1. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  2. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  3. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  4. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  5. 2019Large Scale Incremental Learningcited by 727
  6. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  7. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  8. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  9. 2017Deep Reinforcement Learning: An Overviewcited by 327
  10. 2019Challenges of Real-World Reinforcement Learningcited by 282
  11. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  12. 2020Approximate Data Deletion from Machine Learning Modelscited by 250
38 more
  1. 2023Class-Incremental Learning: A Surveycited by 209
  2. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 175
  3. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  4. 2021Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuningcited by 145
  5. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidationcited by 113
  6. 2020A Comprehensive Study of Class Incremental Learning Algorithms for Visual Taskscited by 107
  7. 2022Federated Unlearning: How to Efficiently Erase a Client in FL?cited by 90
  8. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Spacecited by 88
  9. 2021Federated Learning: Opportunities and Challengescited by 83
  10. 2024Continual Learning and Catastrophic Forgettingcited by 73
  11. 2020Incremental Learning In Online Scenariocited by 71
  12. 2021SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learningcited by 71
  13. 2022Recommendation Unlearningcited by 71
  14. 2022Federated Unlearning with Knowledge Distillationcited by 70
  15. 2020Video Face Manipulation Detection Through Ensemble of CNNscited by 63
  16. 2020Learn to Forget: Machine Unlearning via Neuron Maskingcited by 56
  17. 2018Reinforcement Learning for Improving Agent Designcited by 55
  18. 2021Self-Supervised Training Enhances Online Continual Learningcited by 53
  19. 2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 42
  20. 2022VeriFi: Towards Verifiable Federated Unlearningcited by 41
  21. 2021Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Surveycited by 34
  22. 2020Federated Unlearningcited by 27
  23. 2023Selective and Collaborative Influence Function for Efficient Recommendation Unlearningcited by 25
  24. 2024Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metricscited by 25
  25. 2020Policy Teaching in Reinforcement Learning via Environment Poisoning Attackscited by 24
  26. 2023Recommendation Unlearning via Influence Functioncited by 21
  27. 2018Security and Privacy Issues in Deep Learningcited by 19
  28. 2023Machine Unlearning in Learned Databases: An Experimental Analysiscited by 17
  29. 2024Communication Efficient and Provable Federated Unlearningcited by 16
  30. 2024Unlearning during Learning: An Efficient Federated Machine Unlearning Methodcited by 14
  31. 2024Potion: Towards Poison Unlearningcited by 14
  32. 2023A Review of the Trends and Challenges in Adopting Natural Language Processing Methods for Education Feedback Analysiscited by 12
  33. 2023FRAMU: Attention-based Machine Unlearning using Federated Reinforcement Learningcited by 12
  34. 2023QuickDrop: Efficient Federated Unlearning by Integrated Dataset Distillationcited by 11
  35. 2022Human Activity Recognition Using Tools of Convolutional Neural Networks: A State of the Art Review, Data Sets, Challenges and Future Prospectscited by 10
  36. 2023Elephant Neural Networks: Born to Be a Continual Learnercited by 10
  37. 2020Distributed Deep Reinforcement Learning: An Overviewcited by 9
  38. 2023GraphGuard: Detecting and Counteracting Training Data Misuse in Graph Neural Networkscited by 8

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  2. 2026FG-OrIU: Towards Better Forgetting via Feature-Gradient Orthogonality for Incremental Unlearningcited by 5