# Bayesian Inference Forgetting

arXiv: https://arxiv.org/abs/2101.06417 · HTML: https://mlsanity.com/paper/2101.06417/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 77 references that are in the map.
- First posted on arXiv 16 Jan 2021.

## Builds on (most cited first, 27 of 27)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Approximate Data Deletion from Machine Learning Models](https://mlsanity.com/paper/2002.10077/) (2020)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://mlsanity.com/paper/1902.03932/) (2019)
- [A Complete Recipe for Stochastic Gradient MCMC](https://mlsanity.com/paper/1506.04696/) (2015)
- [Variational Bayesian Unlearning](https://mlsanity.com/paper/2010.12883/) (2020)
- [Preserving Statistical Validity in Adaptive Data Analysis](https://mlsanity.com/paper/1411.2664/) (2014)
- [Machine Unlearning: Linear Filtration for Logit-based Classifiers](https://mlsanity.com/paper/2002.02730/) (2020)
- [Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't](https://mlsanity.com/paper/2009.10713/) (2020)
- [Uncertainty in Neural Networks: Approximately Bayesian Ensembling](https://mlsanity.com/paper/1810.05546/) (2018)
- [Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network](https://mlsanity.com/paper/1810.01279/) (2018)
- [Formalizing Data Deletion in the Context of the Right to be Forgotten](https://mlsanity.com/paper/2002.10635/) (2020)
- [Towards Probabilistic Verification of Machine Unlearning](https://mlsanity.com/paper/2003.04247/) (2020)
- [Recent advances in deep learning theory](https://mlsanity.com/paper/2012.10931/) (2020)
- [Tighter Generalization Bounds for Iterative Differentially Private Learning Algorithms](https://mlsanity.com/paper/2007.09371/) (2020)
- [Robust Online Matrix Factorization for Dynamic Background Subtraction](https://mlsanity.com/paper/1705.10000/) (2017)
- [Optimizing regularized Cholesky score for order-based learning of Bayesian networks](https://mlsanity.com/paper/1904.12360/) (2019)
- [Deep Autoencoding Topic Model with Scalable Hybrid Bayesian Inference](https://mlsanity.com/paper/2006.08804/) (2020)

## Built on by (most cited first, 7 of 7)

- [A Survey of Machine Unlearning](https://mlsanity.com/paper/2209.02299/) (2022)
- [Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning](https://mlsanity.com/paper/2401.10371/) (2024)
- [Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten](https://mlsanity.com/paper/2202.13585/) (2022)
- [Few-Shot Unlearning by Model Inversion](https://mlsanity.com/paper/2205.15567/) (2022)
- [Forget-SVGD: Particle-Based Bayesian Federated Unlearning](https://mlsanity.com/paper/2111.12056/) (2021)
- [Task-Aware Machine Unlearning and Its Application in Load Forecasting](https://mlsanity.com/paper/2308.14412/) (2023)
- [How to beat a Bayesian adversary](https://mlsanity.com/paper/2407.08678/) (2024)

## Abstract (arXiv)

The right to be forgotten has been legislated in many countries but the enforcement in machine learning would cause unbearable costs: companies may need to delete whole models learned from massive resources due to single individual requests. Existing works propose to remove the knowledge learned from the requested data via its influence function which is no longer naturally well-defined in Bayesian inference. This paper proposes a it Bayesian inference forgetting (BIF) framework to realize the right to be forgotten in Bayesian inference. In the BIF framework, we develop forgetting algorithms for variational inference and Markov chain Monte Carlo. We show that our algorithms can provably remove the influence of single datums on the learned models. Theoretical analysis demonstrates that our algorithms have guaranteed generalizability. Experiments of Gaussian mixture models on the synthetic dataset and Bayesian neural networks on the real-world data verify the feasibility of our methods. The source code package is available at urlhttps://github.com/fshp971/BIF.