mlsanityData to 7 Oct 2026

Unlearning with Fisher Masking

arXiv 2310.05331 ↗First posted 9 Oct 2023cs.LGcited by 8

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning aims to revoke some training data after learning in response to requests from users, model developers, and administrators. Most previous methods are based on direct fine-tuning, which may neither remove data completely nor retain full performances on the remain data. In this work, we find that, by first masking some important parameters before fine-tuning, the performances of unlearning could be significantly improved. We propose a new masking strategy tailored to unlearning based on Fisher information. Experiments on various datasets and network structures show the effectiveness of the method: without any fine-tuning, the proposed Fisher masking could unlearn almost completely while maintaining most of the performance on the remain data. It also exhibits stronger stability compared to other unlearning baselines

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 28 of its 42 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

28 of 42 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2014Going Deeper with Convolutionscited by 4,231
  6. 2016Deep Learning with Differential Privacycited by 2,682
  7. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  8. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  9. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  10. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  11. 2019Certified Data Removal from Machine Learning Modelscited by 512
  12. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
16 more
  1. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  2. 2018Learning Not to Learn: Training Deep Neural Networks with Biased Datacited by 255
  3. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  4. 2020Approximate Data Deletion from Machine Learning Modelscited by 250
  5. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Triggercited by 203
  6. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 146
  7. 2018Memory Efficient Experience Replay for Streaming Learningcited by 128
  8. 2020Machine Unlearning for Random Forestscited by 125
  9. 2021Federated Unlearning via Class-Discriminative Pruningcited by 108
  10. 2020Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identificationcited by 80
  11. 2020Generalized Variational Continual Learningcited by 56
  12. 2021Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?cited by 52
  13. 2020InsideBias: Measuring Bias in Deep Networks and Application to Face Gender Biometricscited by 31
  14. 2021Dynamic Differential-Privacy Preserving SGDcited by 29
  15. 2021Network Pruning That Matters: A Case Study on Retraining Variantscited by 25
  16. 2020LOGAN: Local Group Bias Detection by Clusteringcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 4; 2026: 1.

8 papers.

  1. 2023DeepClean: Machine Unlearning on the Cheap by Resetting Privacy Sensitive Weights using the Fisher Diagonalcited by 7
  2. 2025Targeted Forgetting of Image Subgroups in CLIP Modelscited by 6
  3. 2024Attack and Reset for Unlearning: Exploiting Adversarial Noise toward Machine Unlearning through Parameter Re-initializationcited by 5
  4. 2025Exploring Nonlinear Pathway in Parameter Space for Machine Unlearningcited by 4
  5. 2025Module-Aware Parameter-Efficient Machine Unlearning on Transformerscited by 3
  6. 2024Discriminative Adversarial Unlearningcited by 2
  7. 2025Sharpness-Aware Parameter Selection for Machine Unlearningcited by 2
  8. 2026Open Vocabulary Domain Unlearningcited by 0