mlsanityData to 7 Oct 2026

A unified PAC-Bayesian framework for machine unlearning via information risk minimization

arXiv 2106.00265 ↗First posted 1 Jun 2021cs.LG cs.IT eess.SPcited by 6

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning refers to mechanisms that can remove the influence of a subset of training data upon request from a trained model without incurring the cost of re-training from scratch. This paper develops a unified PAC-Bayesian framework for machine unlearning that recovers the two recent design principles - variational unlearning (Nguyen et.al., 2020) and forgetting Lagrangian (Golatkar et.al., 2020) - as information risk minimization problems (Zhang,2006). Accordingly, both criteria can be interpreted as PAC-Bayesian upper bounds on the test loss of the unlearned model that take the form of free energy metrics.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 9 of its 17 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

9 of 17 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar

Other references · 8

  1. 2019Machine Unlearningcited by 1,020
  2. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  3. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  4. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  5. 2019A Primer on PAC-Bayesian Learningcited by 130
  6. 2020Variational Bayesian Unlearningcited by 114
  7. 2017Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priorscited by 81
  8. 2020PAC-Bayes Analysis Beyond the Usual Boundscited by 63

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 1; 2024: 2; 2025: 1.

6 papers.

  1. 2022A Survey of Machine Unlearningcited by 298
  2. 2024A Review on Machine Unlearningcited by 30
  3. 2021Forget-SVGD: Particle-Based Bayesian Federated Unlearningcited by 14
  4. 2024Unlearning during Learning: An Efficient Federated Machine Unlearning Methodcited by 14
  5. 2025Deep Contrastive Unlearning for Language Modelscited by 4
  6. 2023CovarNav: Machine Unlearning via Model Inversion and Covariance Navigationcited by 2