Wasserstein PAC-Bayes Learning: Exploiting Optimisation Guarantees to Explain Generalisation
Not on the map. Why?
Abstract (arXiv)
PAC-Bayes learning is an established framework to both assess the generalisation ability of learning algorithms, and design new learning algorithm by exploiting generalisation bounds as training objectives. Most of the exisiting bounds involve a Kullback-Leibler (KL) divergence, which fails to capture the geometric properties of the loss function which are often useful in optimisation. We address this by extending the emerging Wasserstein PAC-Bayes theory. We develop new PAC-Bayes bounds with Wasserstein distances replacing the usual KL, and demonstrate that sound optimisation guarantees translate to good generalisation abilities. In particular we provide generalisation bounds for the Bures-Wasserstein SGD by exploiting its optimisation properties.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 36 of its 57 references that are in the map. First posted on arXiv 14 Apr 2023.
Builds on
36 of 57 references.
References · 36
- 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
- 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
- 2021User-friendly introduction to PAC-Bayes boundscited by 234
- 2019A Primer on PAC-Bayesian Learningcited by 130
- 2015On the properties of variational approximations of Gibbs posteriorscited by 121
- 2020Tighter risk certificates for neural networkscited by 112
- 2020PACOH: Bayes-Optimal Meta-Learning with PAC-Guaranteescited by 85
- 2022Variational inference via Wasserstein gradient flowscited by 84
- 2017Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priorscited by 81
- 2015Generalization in Adaptive Data Analysis and Holdout Reusecited by 76
- 2017Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theorycited by 76
- 2020PAC-Bayes Analysis Beyond the Usual Boundscited by 63
24 more
- 2016Simpler PAC-Bayesian Bounds for Hostile Datacited by 45
- 2020PAC-Bayes unleashed: generalisation bounds with unbounded lossescited by 43
- 2021Tighter expected generalization error bounds via Wasserstein distancecited by 43
- 2019Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networkscited by 38
- 2019PAC-Bayes Un-Expected Bernstein Inequalitycited by 36
- 2019PAC-Bayes with Backpropcited by 36
- 2021Averaging on the Bures-Wasserstein manifold: dimension-free convergence of gradient descentcited by 32
- 2020Differentiable PAC-Bayes Objectives with Partially Aggregated Neural Networkscited by 29
- 2021Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stabilitycited by 29
- 2016Max-Information, Differential Privacy, and Post-Selection Hypothesis Testingcited by 25
- 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 23
- 2022Non-Vacuous Generalisation Bounds for Shallow Neural Networkscited by 22
- 2022Online PAC-Bayes Learningcited by 22
- 2021Bridging the Gap Between Practice and PAC-Bayes Theory in Few-Shot Meta-Learningcited by 21
- 2021Learning PAC-Bayes Priors for Probabilistic Neural Networkscited by 21
- 2022Integral Probability Metrics PAC-Bayes Boundscited by 19
- 2019PAC-Bayes under potentially heavy tailscited by 16
- 2019Efron-Stein PAC-Bayesian Inequalitiescited by 15
- 2023Tighter PAC-Bayes Bounds Through Coin-Bettingcited by 13
- 2023PAC-Bayesian Generalization Bounds for Adversarial Generative Modelscited by 13
- 2021Progress in Self-Certified Neural Networkscited by 10
- 2022Change of measure through the Legendre transformcited by 10
- 2023Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayescited by 8
- 2022Shedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distancescited by 7
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.