mlsanityData to 7 Oct 2026

Continual Learning and Private Unlearning

arXiv 2203.12817 ↗First posted 24 Mar 2022cs.AIcited by 124

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

As intelligent agents become autonomous over longer periods of time, they may eventually become lifelong counterparts to specific people. If so, it may be common for a user to want the agent to master a task temporarily but later on to forget the task due to privacy concerns. However enabling an agent to forget privately what the user specified without degrading the rest of the learned knowledge is a challenging problem. With the aim of addressing this challenge, this paper formalizes this continual learning and private unlearning (CLPU) problem. The paper further introduces a straightforward but exactly private solution, CLPU-DER++, as the first step towards solving the CLPU problem, along with a set of carefully designed benchmark problems to evaluate the effectiveness of the proposed solution. The code is available at https://github.com/Cranial-XIX/Continual-Learning-Private-Unlearning.

Key facts. Cited by 124 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 26 of its 38 references that are in the map. First posted on arXiv 24 Mar 2022.

Builds on

26 of 38 references.

References · 26

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2016Learning without Forgettingcited by 2,465
  3. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  4. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  5. 2016Progressive Neural Networkscited by 1,332
  6. 2017Continual Learning with Deep Generative Replaycited by 1,152
  7. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  8. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  9. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  10. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  11. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  12. 2019Three scenarios for continual learningcited by 610
14 more
  1. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  2. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  3. 2019Certified Data Removal from Machine Learning Modelscited by 512
  4. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  5. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  6. 2018Task-Free Continual Learningcited by 219
  7. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  8. 2020Mixed-Privacy Forgetting in Deep Networkscited by 158
  9. 2020Continual Lifelong Learning in Natural Language Processing: A Surveycited by 152
  10. 2020DeltaGrad: Rapid retraining of machine learning modelscited by 143
  11. 2020Machine Unlearning for Random Forestscited by 125
  12. 2017Incremental Learning Through Deep Adaptationcited by 108
  13. 2021Firefly Neural Architecture Descent: a General Approach for Growing Neural Networkscited by 48
  14. 2018Security and Privacy Issues in Deep Learningcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 20; 2025: 53; 2026: 46.

124 papers.

Sort:

By area:

  1. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  2. 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 507
  3. 2022A Survey of Machine Unlearningcited by 298
  4. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  5. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearningcited by 81
  6. 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 76
  7. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyondcited by 76
  8. 2024LLM Unlearning via Loss Adjustment with Only Forget Datacited by 71
  9. 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Benchcited by 61
  10. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancementcited by 43
  11. 2024Evaluating Copyright Takedown Methods for Language Modelscited by 42
  12. 2025LLM Unlearning via Neural Activation Redirectioncited by 38
38 more
  1. 2024Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMscited by 36
  2. 2024CLEAR: Character Unlearning in Textual and Visual Modalitiescited by 36
  3. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
  4. 2023The Ideal Continual Learner: An Agent That Never Forgetscited by 34
  5. 2024Offset Unlearning for Large Language Modelscited by 33
  6. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Modelscited by 30
  7. 2024Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning ratecited by 29
  8. 2024WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Modelscited by 25
  9. 2025Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Modelscited by 25
  10. 2025BLUR: A Bi-Level Optimization Approach for LLM Unlearningcited by 23
  11. 2024A Probabilistic Perspective on Unlearning and Alignment for Large Language Modelscited by 21
  12. 2023MultiDelete for Multimodal Machine Unlearningcited by 20
  13. 2024Incremental Learning of Retrievable Skills For Efficient Continual Task Adaptationcited by 19
  14. 2025GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detectioncited by 17
  15. 2024MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearningcited by 16
  16. 2025SAUCE: Selective Concept Unlearning in Vision-Language Models with Sparse Autoencoderscited by 12
  17. 2023Gradient Surgery for One-shot Unlearning on Generative Modelcited by 11
  18. 2026Shaping capabilities with token-level data filteringcited by 11
  19. 2025SoK: Machine Unlearning for Large Language Modelscited by 9
  20. 2025UCD: Unlearning in LLMs via Contrastive Decodingcited by 9
  21. 2025MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steeringcited by 9
  22. 2025Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decodingcited by 9
  23. 2025Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspectivecited by 8
  24. 2024Federated Unlearning: a Perspective of Stability and Fairnesscited by 7
  25. 2025Does Machine Unlearning Truly Remove Knowledge?cited by 7
  26. 2025Speech Unlearningcited by 7
  27. 2025Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMscited by 7
  28. 2025CLUE: Conflict-guided Localization for LLM Unlearning Frameworkcited by 7
  29. 2025OFFSIDE: Benchmarking Unlearning Misinformation in Multimodal Large Language Modelscited by 7
  30. 2025Cross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Modelscited by 7
  31. 2025PEBench: A Fictitious Dataset to Benchmark Machine Unlearning for Multimodal Large Language Modelscited by 6
  32. 2025A Unified Gradient-based Framework for Task-agnostic Continual Learning-Unlearningcited by 6
  33. 2025Not All Tokens Are Meant to Be Forgottencited by 6
  34. 2025Improving Fisher Information Estimation and Efficiency for LoRA-based LLM Unlearningcited by 6
  35. 2025On the Impossibility of Retrain Equivalence in Machine Unlearningcited by 6
  36. 2025Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearningcited by 6
  37. 2026Multilingual Amnesia: On the Transferability of Unlearning in Multilingual LLMscited by 5
  38. 2026FG-OrIU: Towards Better Forgetting via Feature-Gradient Orthogonality for Incremental Unlearningcited by 5

50 most cited of 124 All 124 (JSON)