# Forget-SVGD: Particle-Based Bayesian Federated Unlearning

arXiv: https://arxiv.org/abs/2111.12056 · HTML: https://mlsanity.com/paper/2111.12056/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 15 of its 18 references that are in the map.
- First posted on arXiv 23 Nov 2021.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 15 of 15)

- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Remember What You Want to Forget: Algorithms for Machine Unlearning](https://mlsanity.com/paper/2103.03279/) (2021)
- [Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks](https://mlsanity.com/paper/2002.10118/) (2020)
- [On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning](https://mlsanity.com/paper/2110.11891/) (2021)
- [Variational Bayesian Unlearning](https://mlsanity.com/paper/2010.12883/) (2020)
- [Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descent](https://mlsanity.com/paper/2009.06419/) (2020)
- [Partitioned Variational Inference: A unified framework encompassing federated and continual learning](https://mlsanity.com/paper/1811.11206/) (2018)
- [QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning](https://mlsanity.com/paper/2106.00797/) (2021)
- [Bayesian Variational Federated Learning and Unlearning in Decentralized Networks](https://mlsanity.com/paper/2104.03834/) (2021)
- [Bayesian Inference Forgetting](https://mlsanity.com/paper/2101.06417/) (2021)
- [A unified PAC-Bayesian framework for machine unlearning via information risk minimization](https://mlsanity.com/paper/2106.00265/) (2021)
- [Bayesian Modelling in Practice: Using Uncertainty to Improve Trustworthiness in Medical Applications](https://mlsanity.com/paper/1906.08619/) (2019)

## Built on by (most cited first, 16 of 16)

- [A Survey of Machine Unlearning](https://mlsanity.com/paper/2209.02299/) (2022)
- [A Survey on Federated Unlearning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/2310.20448/) (2023)
- [Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2305.06360/) (2023)
- [A Review on Machine Unlearning](https://mlsanity.com/paper/2411.11315/) (2024)
- [SAFE: Machine Unlearning With Shard Graphs](https://mlsanity.com/paper/2304.13169/) (2023)
- [Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics](https://mlsanity.com/paper/2401.05146/) (2024)
- [Machine Unlearning: A Survey](https://mlsanity.com/paper/2306.03558/) (2023)
- [SIFU: Sequential Informed Federated Unlearning for Efficient and Provable Client Unlearning in Federated Optimization](https://mlsanity.com/paper/2211.11656/) (2022)
- [A Survey on Machine Unlearning: Techniques and New Emerged Privacy Risks](https://mlsanity.com/paper/2406.06186/) (2024)
- [Get Rid Of Your Trail: Remotely Erasing Backdoors in Federated Learning](https://mlsanity.com/paper/2304.10638/) (2023)
- [Compressed Particle-Based Federated Bayesian Learning and Unlearning](https://mlsanity.com/paper/2209.07267/) (2022)
- [Vertical Federated Unlearning via Backdoor Certification](https://mlsanity.com/paper/2412.11476/) (2024)
- [On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directions](https://mlsanity.com/paper/2306.01431/) (2023)
- [A Survey on Federated Unlearning: Challenges and Opportunities](https://mlsanity.com/paper/2403.02437/) (2024)
- [Streamlined Federated Unlearning: Unite as One to Be Highly Efficient](https://mlsanity.com/paper/2412.00126/) (2024)
- [Federated Unlearning Model Recovery in Data with Skewed Label Distributions](https://mlsanity.com/paper/2412.13466/) (2024)

## Abstract (arXiv)

Variational particle-based Bayesian learning methods have the advantage of not being limited by the bias affecting more conventional parametric techniques. This paper proposes to leverage the flexibility of non-parametric Bayesian approximate inference to develop a novel Bayesian federated unlearning method, referred to as Forget-Stein Variational Gradient Descent (Forget-SVGD). Forget-SVGD builds on SVGD - a particle-based approximate Bayesian inference scheme using gradient-based deterministic updates - and on its distributed (federated) extension known as Distributed SVGD (DSVGD). Upon the completion of federated learning, as one or more participating agents request for their data to be "forgotten", Forget-SVGD carries out local SVGD updates at the agents whose data need to be "unlearned", which are interleaved with communication rounds with a parameter server. The proposed method is validated via performance comparisons with non-parametric schemes that train from scratch by excluding data to be forgotten, as well as with existing parametric Bayesian unlearning methods.