mlsanityData to 14 Oct 2026

Evaluating Machine Unlearning via Epistemic Uncertainty

arXiv 2208.10836 ↗First posted 23 Aug 2022cs.LGcited by 39

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

There has been a growing interest in Machine Unlearning recently, primarily due to legal requirements such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act. Thus, multiple approaches were presented to remove the influence of specific target data points from a trained model. However, when evaluating the success of unlearning, current approaches either use adversarial attacks or compare their results to the optimal solution, which usually incorporates retraining from scratch. We argue that both ways are insufficient in practice. In this work, we present an evaluation metric for Machine Unlearning algorithms based on epistemic uncertainty. This is the first definition of a general evaluation metric for Machine Unlearning to our best knowledge.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 23 of its 32 references that are in the map. First posted on arXiv 23 Aug 2022.

Builds on

23 of 32 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Knowledge Removal in Sampling-based Bayesian Inference24 Mar
  2. 2022Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten28 Feb
  3. 2021Machine Unlearning of Features and Labels26 Aug
  4. 2021Adaptive Machine Unlearning8 Jun
  5. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar
  6. 2021Machine Unlearning via Algorithmic Stability25 Feb

Other references · 17

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  3. 2019Machine Unlearningcited by 1,061
  4. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  5. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 618
  6. 2019Certified Data Removal from Machine Learning Modelscited by 535
  7. 2014New insights and perspectives on the natural gradient methodcited by 443
  8. 2020Amnesiac Machine Learningcited by 319
  9. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 311
  10. 2018Adversarial Robustness Toolbox v1.0.0cited by 215
  11. 2020Mixed-Privacy Forgetting in Deep Networkscited by 163
  12. 2020Machine Unlearning for Random Forestscited by 131
5 more
  1. 2021Federated Unlearning via Class-Discriminative Pruningcited by 115
  2. 2022Recommendation Unlearningcited by 80
  3. 2022Backdoor Defense with Machine Unlearningcited by 65
  4. 2020Coded Machine Unlearningcited by 36
  5. 2022Efficiently Maintaining Next Basket Recommendations under Additions and Deletions of Baskets and Itemscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 19; 2025: 12; 2026: 2.

39 papers.

Sort:

By area:

  1. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 386
  2. 2022A Survey of Machine Unlearningcited by 312
  3. 2024Rethinking Machine Unlearning for Large Language Modelscited by 257
  4. 2023Model Sparsity Can Simplify Machine Unlearningcited by 214
  5. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 210
  6. 2024LLM Unlearning via Loss Adjustment with Only Forget Datacited by 75
  7. 2024Scissorhands: Scrub Data Influence via Connection Sensitivity in Networkscited by 52
  8. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 51
  9. 2024UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Modelscited by 50
  10. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearningcited by 44
  11. 2024Machine Unlearning: A Comprehensive Surveycited by 35
  12. 2024WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Modelscited by 26
27 more
  1. 2024Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Modelscited by 23
  2. 2023Breaking the Trilemma of Privacy, Utility, Efficiency via Controllable Machine Unlearningcited by 22
  3. 2024Towards Robust Evaluation of Unlearning in LLMs via Data Transformationscited by 20
  4. 2024An Information Theoretic Evaluation Metric For Strong Unlearningcited by 11
  5. 2024Label Smoothing Improves Machine Unlearningcited by 9
  6. 2025Towards Reliable Forgetting: A Survey on Machine Unlearning Verificationcited by 8
  7. 2025Go Beyond Your Means: Unlearning with Per-Sample Gradient Orthogonalizationcited by 7
  8. 2024Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Approximate Unlearning Completenesscited by 6
  9. 2024Forget Vectors at Play: Universal Input Perturbations Driving Machine Unlearning in Image Classificationcited by 6
  10. 2025Targeted Forgetting of Image Subgroups in CLIP Modelscited by 6
  11. 2025Do Not Mimic My Voice: Speaker Identity Unlearning for Zero-Shot Text-to-Speechcited by 6
  12. 2024A Reliable Cryptographic Framework for Empirical Machine Unlearning Evaluationcited by 5
  13. 2024Understanding Fine-tuning in Approximate Unlearning: A Theoretical Perspectivecited by 5
  14. 2025Efficient Utility-Preserving Machine Unlearning with Implicit Gradient Surgerycited by 5
  15. 2024On the Impact of Uncertainty and Calibration on Likelihood-Ratio Membership Inference Attackscited by 4
  16. 2025Unlearning Comparator: A Visual Analytics System for Comparative Evaluation of Machine Unlearning Methodscited by 4
  17. 2025Tackling Fake Forgetting through Uncertainty Quantificationcited by 3
  18. 2024Discriminative Adversarial Unlearningcited by 2
  19. 2024Unlearning Information Bottleneck: Machine Unlearning of Systematic Patterns and Biasescited by 2
  20. 2024Silver Linings in the Shadows: Harnessing Membership Inference for Machine Unlearningcited by 2
  21. 2025Recalling The Forgotten Class Memberships: Unlearned Models Can Be Noisy Labelers to Leak Privacycited by 1
  22. 2025Invisible Watermarks, Visible Gains: Steering Machine Unlearning with Bi-Level Watermarking Designcited by 1
  23. 2025CUFG: Curriculum Unlearning Guided by the Forgetting Gradientcited by 1
  24. 2025WaterDrum: Watermarking for Data-centric Unlearning Metriccited by 0
  25. 2025COLUR: Confidence-Oriented Learning, Unlearning and Relearning with Noisy-Label Data for Model Restoration and Refinementcited by 0
  26. 2026Spectral Saliency for Machine Unlearningcited by 0
  27. 2026PruneForget: Joint Unlearning and Pruning of Vision Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspective
  2. 2024Gone but Not Forgotten: Improved Benchmarks for Machine Unlearning
  3. 2024Verification of Machine Unlearning is Fragile
  4. 2023Learn to Unlearn: A Survey on Machine Unlearning
  5. 2024Machine unlearning through fine-grained model parameters perturbation
  6. 2026Unlearning Evaluation through Subset Statistical Independence
  7. 2022Towards Adversarial Evaluations for Inexact Machine Unlearning
  8. 2025Are We Truly Forgetting? A Critical Re-examination of Machine Unlearning Evaluation Protocols