mlsanityData to 7 Oct 2026

Learning to Unlearn for Robust Machine Unlearning

arXiv 2407.10494 ↗First posted 15 Jul 2024cs.LG cs.CVcited by 19

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning (MU) seeks to remove knowledge of specific data samples from trained models without the necessity for complete retraining, a task made challenging by the dual objectives of effective erasure of data and maintaining the overall performance of the model. Despite recent advances in this field, balancing between the dual objectives of unlearning remains challenging. From a fresh perspective of generalization, we introduce a novel Learning-to-Unlearn (LTU) framework, which adopts a meta-learning approach to optimize the unlearning process to improve forgetting and remembering in a unified manner. LTU includes a meta-optimization scheme that facilitates models to effectively preserve generalizable knowledge with only a small subset of the remaining set, while thoroughly forgetting the specific data samples. We also introduce a Gradient Harmonization strategy to align the optimization trajectories for remembering and forgetting via mitigating gradient conflicts, thus ensuring efficient and effective model updates. Our approach demonstrates improved efficiency and efficacy for MU, offering a promising solution to the challenges of data rights and model reusability.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 51 of its 66 references that are in the map. First posted on arXiv 15 Jul 2024.

Builds on

51 of 66 references.

Closest ancestors · 18

Same direction, newest first

  1. 2024GENIU: A Restricted Data Access Unlearning for Imbalanced Data12 Jun
  2. 2024Adversarial Machine Unlearning11 Jun
  3. 2024Digital Forgetting in Large Language Models: A Survey of Unlearning Methods2 Apr
  4. 2024Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Models31 Mar
  5. 2023Fast Model Debias with Machine Unlearning19 Oct
  6. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation19 Oct
  7. 2023Disposable Transfer Learning for Selective Source Task Unlearning19 Aug
  8. 2023Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening15 Aug
  9. 2023Machine Unlearning: Solutions and Challenges14 Aug
  10. 2023SAFE: Machine Unlearning With Shard Graphs25 Apr
  11. 2023Boundary Unlearning21 Mar
  12. 2023Towards Unbounded Machine Unlearning20 Feb
6 more
  1. 2023Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers27 Jan
  2. 2022Forget Unlearning: Towards True Data-Deletion in Machine Learning17 Oct
  3. 2022Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher17 May
  4. 2022Deep Unlearning via Randomized Conditionally Independent Hessians15 Apr
  5. 2021Fast Yet Effective Machine Unlearning17 Nov
  6. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep

Other references · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  6. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  7. 2019Machine Unlearningcited by 1,020
  8. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  9. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  10. 2019Meta-Learning with Implicit Gradientscited by 523
  11. 2019Certified Data Removal from Machine Learning Modelscited by 512
  12. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
21 more
  1. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  2. 2020Amnesiac Machine Learningcited by 305
  3. 2020When Machine Unlearning Jeopardizes Privacycited by 181
  4. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 175
  5. 2020Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observationscited by 171
  6. 2020Mixed-Privacy Forgetting in Deep Networkscited by 158
  7. 2020DeltaGrad: Rapid retraining of machine learning modelscited by 143
  8. 2020Variational Bayesian Unlearningcited by 114
  9. 2022Membership Inference Attacks by Exploiting Loss Trajectorycited by 93
  10. 2020Learning Meta Face Recognition in Unseen Domainscited by 61
  11. 2024Machine Unlearning for Image-to-Image Generative Modelscited by 47
  12. 2023GNNDelete: A General Strategy for Unlearning in Graph Neural Networkscited by 43
  13. 2022Learning with Recoverable Forgettingcited by 29
  14. 2021SSSE: Efficiently Erasing Samples from Trained Machine Learning Modelscited by 28
  15. 2022MetaSets: Meta-Learning on Point Sets for Generalizable Representationscited by 22
  16. 2023Meta Compositional Referring Expression Segmentationcited by 18
  17. 2022Distilling Inter-Class Distance for Semantic Segmentationcited by 16
  18. 2022ERA: Expert Retrieval and Assembly for Early Action Predictioncited by 9
  19. 2023Audit to Forget: A Unified Method to Revoke Patients' Private Data in Intelligent Healthcarecited by 9
  20. 2023System-status-aware Adaptive Network for Online Streaming Video Understandingcited by 9
  21. 2023GradMDM: Adversarial Attack on Dynamic Networkscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 11; 2026: 5.

19 papers.

Sort:

By area:

  1. 2024Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Conceptscited by 39
  2. 2025Distillation Robustifies Unlearningcited by 11
  3. 2025Go Beyond Your Means: Unlearning with Per-Sample Gradient Orthogonalizationcited by 7
  4. 2025Targeted Forgetting of Image Subgroups in CLIP Modelscited by 6
  5. 2025Efficient Utility-Preserving Machine Unlearning with Implicit Gradient Surgerycited by 5
  6. 2024Unlearning Personal Data from a Single Imagecited by 4
  7. 2025Group-robust Machine Unlearningcited by 3
  8. 2026Models Designed to Forget: Machine Unlearning via Key Deletioncited by 3
  9. 2024ConDa: Fast Federated Unlearning with Contribution Dampeningcited by 2
  10. 2025Exploring Incremental Unlearning: Techniques, Challenges, and Future Directionscited by 2
  11. 2025Ready2Unlearn: A Learning-Time Approach for Preparing Models with Future Unlearning Readinesscited by 2
  12. 2025Towards Efficient and Exact Forgetting Services in Pre-Trained-Model-based Continual Learningcited by 2
7 more
  1. 2025CUFG: Curriculum Unlearning Guided by the Forgetting Gradientcited by 1
  2. 2025Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimizationcited by 1
  3. 2025REMISVFU: Vertical Federated Unlearning via Representation Misdirection for Intermediate Output Featurecited by 1
  4. 2026Unlearning the Unpromptable: Prompt-free Instance Unlearning in Diffusion Modelscited by 1
  5. 2026Jellyfish: Zero-Shot Federated Unlearning Scheme with Knowledge Disentanglementcited by 1
  6. 2026Retain-Neutral Surrogates for Min-Max Unlearningcited by 0
  7. 2026Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearningcited by 0