mlsanityData to 14 Oct 2026

Stability-based Generalization Bounds for Variational Inference

arXiv 2502.12353 ↗First posted 17 Feb 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Variational inference (VI) is widely used for approximate inference in Bayesian machine learning. In addition to this practical success, generalization bounds for variational inference and related algorithms have been developed, mostly through the connection to PAC-Bayes analysis. A second line of work has provided algorithm-specific generalization bounds through stability arguments or using mutual information bounds, and has shown that the bounds are tight in practice, but unfortunately these bounds do not directly apply to approximate Bayesian algorithms. This paper fills this gap by developing algorithm-specific stability based generalization bounds for a class of approximate Bayesian algorithms that includes VI, specifically when using stochastic gradient descent to optimize their objective. As in the non-Bayesian case, the generalization error is bounded by by expected parameter differences on a perturbed dataset. The new approach complements PAC-Bayes analysis and can provide tighter bounds in some cases. An experimental illustration shows that the new approach yields non-vacuous bounds on modern neural network architectures and datasets and that it can shed light on performance differences between variant approximate Bayesian algorithms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 43 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

19 of 43 references.

References · 19

  1. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  2. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 552
  3. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  4. 2017Size-Independent Sample Complexity of Neural Networkscited by 290
  5. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
  6. 2019Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimatescited by 119
  7. 2016Composing graphical models with neural networks for structured representations and fast inferencecited by 113
  8. 2020Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithmscited by 89
  9. 2016PAC-Bayesian Theory Meets Bayesian Inferencecited by 88
  10. 2021Information-Theoretic Generalization Bounds for Stochastic Gradient Descentcited by 84
  11. 2019On Generalization Error Bounds of Noisy Gradient Methods for Non-Convex Learningcited by 57
  12. 2019Generalized Variational Inference: Three arguments for deriving new Posteriorscited by 57
7 more
  1. 2015Black-box α-divergence Minimizationcited by 37
  2. 2022Integral Probability Metrics PAC-Bayes Boundscited by 19
  3. 2019Improved PAC-Bayesian Bounds for Linear Regressioncited by 12
  4. 2023Learning via Wasserstein-Based High Probability Generalisation Boundscited by 12
  5. 2020Direct loss minimization algorithms for sparse Gaussian processescited by 10
  6. 2022Stability Based Generalization Bounds for Exponential Family Langevin Dynamicscited by 8
  7. 2022On the Performance of Direct Loss Minimization for Bayesian Neural Networkscited by 3

Built on by

Not cited yet.