mlsanityData to 14 Oct 2026

Forget-SVGD: Particle-Based Bayesian Federated Unlearning

arXiv 2111.12056 ↗First posted 23 Nov 2021cs.LG eess.SPcited by 16

On the map: LLM safety and security → Federated unlearning and poisoning

Abstract (arXiv)

Variational particle-based Bayesian learning methods have the advantage of not being limited by the bias affecting more conventional parametric techniques. This paper proposes to leverage the flexibility of non-parametric Bayesian approximate inference to develop a novel Bayesian federated unlearning method, referred to as Forget-Stein Variational Gradient Descent (Forget-SVGD). Forget-SVGD builds on SVGD - a particle-based approximate Bayesian inference scheme using gradient-based deterministic updates - and on its distributed (federated) extension known as Distributed SVGD (DSVGD). Upon the completion of federated learning, as one or more participating agents request for their data to be "forgotten", Forget-SVGD carries out local SVGD updates at the agents whose data need to be "unlearned", which are interleaved with communication rounds with a parameter server. The proposed method is validated via performance comparisons with non-parametric schemes that train from scratch by excluding data to be forgotten, as well as with existing parametric Bayesian unlearning methods.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security. Builds on 15 of its 18 references that are in the map. First posted on arXiv 23 Nov 2021.

Builds on

15 of 18 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Bayesian Variational Federated Learning and Unlearning in Decentralized Networks8 Apr

Other references · 14

  1. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 618
  2. 2019Certified Data Removal from Machine Learning Modelscited by 535
  3. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 507
  4. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 465
  5. 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 371
  6. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 231
  7. 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearningcited by 171
  8. 2020Variational Bayesian Unlearningcited by 117
  9. 2020Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descentcited by 35
  10. 2018Partitioned Variational Inference: A unified framework encompassing federated and continual learningcited by 29
  11. 2021QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learningcited by 26
  12. 2021Bayesian Inference Forgettingcited by 7
2 more
  1. 2021A unified PAC-Bayesian framework for machine unlearning via information risk minimizationcited by 6
  2. 2019Bayesian Modelling in Practice: Using Uncertainty to Improve Trustworthiness in Medical Applicationscited by 4

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 6; 2024: 7.

16 papers.

Sort:

By area:

  1. 2022A Survey of Machine Unlearningcited by 312
  2. 2023A Survey on Federated Unlearning: Challenges, Methods, and Future Directionscited by 63
  3. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 51
  4. 2024A Review on Machine Unlearningcited by 32
  5. 2023SAFE: Machine Unlearning With Shard Graphscited by 28
  6. 2024Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metricscited by 27
  7. 2023Machine Unlearning: A Surveycited by 19
  8. 2022SIFU: Sequential Informed Federated Unlearning for Efficient and Provable Client Unlearning in Federated Optimizationcited by 18
  9. 2024A Survey on Machine Unlearning: Techniques and New Emerged Privacy Riskscited by 15
  10. 2023Get Rid Of Your Trail: Remotely Erasing Backdoors in Federated Learningcited by 13
  11. 2022Compressed Particle-Based Federated Bayesian Learning and Unlearningcited by 6
  12. 2024Vertical Federated Unlearning via Backdoor Certificationcited by 6
4 more
  1. 2023On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directionscited by 4
  2. 2024A Survey on Federated Unlearning: Challenges and Opportunitiescited by 0
  3. 2024Streamlined Federated Unlearning: Unite as One to Be Highly Efficientcited by 0
  4. 2024Federated Unlearning Model Recovery in Data with Skewed Label Distributionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Method
  2. 2024Federated Unlearning with Gradient Descent and Conflict Mitigation
  3. 2026f-FUM: Federated Unlearning via min--max and f-divergence
  4. 2023Subspace based Federated Unlearning
  5. 2024ConDa: Fast Federated Unlearning with Contribution Dampening
  6. 2024ADEPT: Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning
  7. 2024Goldfish: An Efficient Federated Unlearning Framework
  8. 2024Communication Efficient and Provable Federated Unlearning