mlsanityData to 13 Oct 2026

Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher

arXiv 2205.08096 ↗First posted 17 May 2022cs.LGcited by 194

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning has become an important area of research due to an increasing need for machine learning (ML) applications to comply with the emerging data privacy regulations. It facilitates the provision for removal of certain set or class of data from an already trained ML model without requiring retraining from scratch. Recently, several efforts have been put in to make unlearning to be effective and efficient. We propose a novel machine unlearning method by exploring the utility of competent and incompetent teachers in a student-teacher framework to induce forgetfulness. The knowledge from the competent and incompetent teachers is selectively transferred to the student to obtain a model that doesn't contain any information about the forget data. We experimentally show that this method generalizes well, is fast and effective. Furthermore, we introduce the zero retrain forgetting (ZRF) metric to evaluate any unlearning method. Unlike the existing unlearning metrics, the ZRF score does not depend on the availability of the expensive retrained model. This makes it useful for analysis of the unlearned model after deployment as well. We present results of experiments conducted for random subset forgetting and class forgetting on various deep networks and across different application domains.~Source code is at: https://github.com/vikram2000b/bad-teaching-unlearning

Key facts. Cited by 194 papers in the map (data through 13 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 23 of its 37 references that are in the map. First posted on arXiv 17 May 2022.

Builds on

23 of 37 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Zero-Shot Machine Unlearning14 Jan
  2. 2021Fast Yet Effective Machine Unlearning17 Nov
  3. 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning22 Oct
  4. 2021Certifiable Machine Unlearning for Linear Models29 Jun
  5. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar
  6. 2021Machine Unlearning via Algorithmic Stability25 Feb

Other references · 17

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  2. 2019Machine Unlearningcited by 1,056
  3. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  4. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 617
  5. 2019Certified Data Removal from Machine Learning Modelscited by 532
  6. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 462
  7. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 456
  8. 2020Amnesiac Machine Learningcited by 319
  9. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 310
  10. 2020Approximate Data Deletion from Machine Learning Modelscited by 255
  11. 2020When Machine Unlearning Jeopardizes Privacycited by 198
  12. 2020Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observationscited by 180
5 more
  1. 2020Mixed-Privacy Forgetting in Deep Networkscited by 163
  2. 2020Machine Unlearning for Random Forestscited by 131
  3. 2020Variational Bayesian Unlearningcited by 116
  4. 2021Federated Unlearning via Class-Discriminative Pruningcited by 115
  5. 2022Federated Unlearning with Knowledge Distillationcited by 73

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 3; 2023: 12; 2024: 67; 2025: 66; 2026: 46.

194 papers.

Sort:

By area:

  1. 2022A Survey of Machine Unlearningcited by 311
  2. 2023Towards Unbounded Machine Unlearningcited by 268
  3. 2023Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampeningcited by 181
  4. 2024Towards Safer Large Language Models through Machine Unlearningcited by 136
  5. 2024Machine Unlearning of Pre-trained Large Language Modelscited by 119
  6. 2023KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignmentcited by 97
  7. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 51
  8. 2024Machine Unlearning in Generative AI: A Surveycited by 49
  9. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancementcited by 44
  10. 2024To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Modelscited by 42
  11. 2024Dissecting Language Models: Machine Unlearning via Selective Pruningcited by 41
  12. 2024Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMscited by 36
38 more
  1. 2024Machine Unlearning: A Comprehensive Surveycited by 35
  2. 2022Deep Regression Unlearningcited by 32
  3. 2024Alternate Preference Optimization for Unlearning Factual Knowledge in Large Language Modelscited by 32
  4. 2024Towards Certified Unlearning for Deep Neural Networkscited by 30
  5. 2024Corrective Machine Unlearningcited by 29
  6. 2025Decoupled Distillation to Erase: A General Unlearning Method for Any Class-centric Taskscited by 28
  7. 2024Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metricscited by 27
  8. 2023Breaking the Trilemma of Privacy, Utility, Efficiency via Controllable Machine Unlearningcited by 22
  9. 2023MultiDelete for Multimodal Machine Unlearningcited by 20
  10. 2024Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Modelscited by 20
  11. 2022Few-Shot Unlearning by Model Inversioncited by 19
  12. 2024Learning to Unlearn for Robust Machine Unlearningcited by 19
  13. 2024Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Modelscited by 18
  14. 2024Is Retain Set All You Need in Machine Unlearning? Restoring Performance of Unlearned Models with Out-Of-Distribution Imagescited by 18
  15. 2024Targeted Unlearning with Single Layer Unlearning Gradientcited by 18
  16. 2024MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearningcited by 17
  17. 2024Towards Efficient and Effective Unlearning of Large Language Models for Recommendationcited by 16
  18. 2024Adversarial Machine Unlearningcited by 16
  19. 2024An Information Theoretic Approach to Machine Unlearningcited by 15
  20. 2024A Survey on Machine Unlearning: Techniques and New Emerged Privacy Riskscited by 15
  21. 2024UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMscited by 15
  22. 2025NoT: Federated Unlearning via Weight Negationcited by 15
  23. 2024Potion: Towards Poison Unlearningcited by 14
  24. 2025Generative Models in Decision Making: A Surveycited by 14
  25. 2023Distill to Delete: Unlearning in Graph Networks with Knowledge Distillationcited by 13
  26. 2023DUCK: Distance-based Unlearning via Centroid Kinematicscited by 13
  27. 2024Remaining-data-free Machine Unlearning by Suppressing Sample Contributioncited by 13
  28. 2024Loss-Free Machine Unlearningcited by 13
  29. 2024Unlearning or Concealment? A Critical Analysis and Evaluation Metrics for Unlearning in Diffusion Modelscited by 13
  30. 2025LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertaintycited by 13
  31. 2025From Dormant to Deleted: Tamper-Resistant Unlearning Through Weight-Space Regularizationcited by 12
  32. 2024nabla τ: Gradient-based and Task-Agnostic machine Unlearningcited by 11
  33. 2024Towards Natural Machine Unlearningcited by 11
  34. 2024Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivitycited by 11
  35. 2024An Information Theoretic Evaluation Metric For Strong Unlearningcited by 11
  36. 2025Distillation Robustifies Unlearningcited by 11
  37. 2024Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampeningcited by 10
  38. 2024Machine Unlearning for Traditional Models and Large Language Models: A Short Surveycited by 10

50 most cited of 194 All 194 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Machine Unlearning Methodology base on Stochastic Teacher Network
  2. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning
  3. 2025CUFG: Curriculum Unlearning Guided by the Forgetting Gradient
  4. 2025When to Forget? Complexity Trade-offs in Machine Unlearning
  5. 2024Machine Unlearning using Forgetting Neural Networks
  6. 2026Why Forget-Only Unlearning Needs Memorization
  7. 2024Machine Unlearning in Forgettability Sequence
  8. 2026Forgetting Has Neighbors: Localized Collateral Forgetting in Machine Unlearning