mlsanityData to 7 Oct 2026

Variational Bayesian Unlearning

arXiv 2010.12883 ↗First posted 24 Oct 2020cs.LG stat.MLcited by 114

Not on the map. Why?

Cited by 92 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper studies the problem of approximately unlearning a Bayesian model from a small subset of the training data to be erased. We frame this problem as one of minimizing the Kullback-Leibler divergence between the approximate posterior belief of model parameters after directly unlearning from erased data vs. the exact posterior belief from retraining with remaining data. Using the variational inference (VI) framework, we show that it is equivalent to minimizing an evidence upper bound which trades off between fully unlearning from erased data vs. not entirely forgetting the posterior belief given the full data (i.e., including the remaining data); the latter prevents catastrophic unlearning that can render the model useless. In model training with VI, only an approximate (instead of exact) posterior belief given the full data can be obtained, which makes unlearning even more challenging. We propose two novel tricks to tackle this challenge. We empirically demonstrate our unlearning methods on Bayesian models such as sparse Gaussian process and logistic regression using synthetic and real-world datasets.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 17; 2023: 21; 2024: 24; 2025: 20; 2026: 20.

114 papers.

Sort:

By area:

  1. 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 360
  2. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
  3. 2022A Survey of Machine Unlearningcited by 298
  4. 2021Fast Yet Effective Machine Unlearningcited by 209
  5. 2022Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teachercited by 189
  6. 2023Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampeningcited by 177
  7. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 175
  8. 2022Zero-Shot Machine Unlearningcited by 125
  9. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  10. 2021Federated Unlearning via Class-Discriminative Pruningcited by 108
  11. 2023Machine Unlearning: Solutions and Challengescited by 85
  12. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
38 more
  1. 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Benchcited by 61
  2. 2024A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trendscited by 58
  3. 2024UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Modelscited by 49
  4. 2024Machine Unlearning in Generative AI: A Surveycited by 49
  5. 2024Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearningcited by 48
  6. 2023GNNDelete: A General Strategy for Unlearning in Graph Neural Networkscited by 43
  7. 2021Certifiable Machine Unlearning for Linear Modelscited by 42
  8. 2022Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attackscited by 37
  9. 2023Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projectioncited by 37
  10. 2022Privacy Adhering Machine Un-learning in NLPcited by 35
  11. 2024Machine Unlearning: A Comprehensive Surveycited by 33
  12. 2022Deep Regression Unlearningcited by 32
  13. 2022Learning with Recoverable Forgettingcited by 29
  14. 2023Certified Minimax Unlearning with Generalization Rates and Deletion Capacitycited by 29
  15. 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Modelscited by 29
  16. 2024Towards Certified Unlearning for Deep Neural Networkscited by 28
  17. 2022Repairing Neural Networks by Leaving the Right Past Behindcited by 27
  18. 2022Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgottencited by 25
  19. 2025Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Modelscited by 25
  20. 2024Data Attribution for Text-to-Image Models by Unlearning Synthesized Imagescited by 22
  21. 2023Deep Unlearning: Fast and Efficient Gradient-free Approach to Class Forgettingcited by 21
  22. 2021Knowledge-Adaptation Priorscited by 20
  23. 2022Knowledge Removal in Sampling-based Bayesian Inferencecited by 20
  24. 2023MultiDelete for Multimodal Machine Unlearningcited by 20
  25. 2024Learning to Unlearn for Robust Machine Unlearningcited by 19
  26. 2023Machine Unlearning: A Surveycited by 18
  27. 2024Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Modelscited by 18
  28. 2022Few-Shot Unlearning by Model Inversioncited by 17
  29. 2024MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearningcited by 16
  30. 2023Feature Unlearning for Pre-trained GANs and VAEscited by 15
  31. 2023ERASER: Machine Unlearning in MLaaS via an Inference Serving-Aware Approachcited by 15
  32. 2021Forget-SVGD: Particle-Based Bayesian Federated Unlearningcited by 14
  33. 2024Contrastive Unlearning: A Contrastive Approach to Machine Unlearningcited by 14
  34. 2025Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?cited by 13
  35. 2023Adapt then Unlearn: Exploring Parameter Space Semantics for Unlearning in Generative Adversarial Networkscited by 11
  36. 2023QuickDrop: Efficient Federated Unlearning by Integrated Dataset Distillationcited by 11
  37. 2024Negating Negatives: Alignment with Human Negative Samples via Distributional Dispreference Optimizationcited by 11
  38. 2023Ticketed Learning-Unlearning Schemescited by 10

50 most cited of 114 All 114 (JSON)