mlsanityData to 7 Oct 2026

Tighter Generalisation Bounds via Interpolation

arXiv 2402.05101 ↗First posted 7 Feb 2024stat.ML cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

This paper contains a recipe for deriving new PAC-Bayes generalisation bounds based on the (f, Gamma)-divergence, and, in addition, presents PAC-Bayes generalisation bounds where we interpolate between a series of probability divergences (including but not limited to KL, Wasserstein, and total variation), making the best out of many worlds depending on the posterior distributions properties. We explore the tightness of these bounds and connect them to earlier results from statistical learning, which are specific cases. We also instantiate our bounds as training objectives, yielding non-trivial guarantees and practical performances.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 36 of its 74 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

36 of 74 references.

References · 36

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  2. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  3. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  4. 2019A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networkscited by 213
  5. 2019A Primer on PAC-Bayesian Learningcited by 130
  6. 2020The Heavy-Tail Phenomenon in SGDcited by 122
  7. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
  8. 2020PACOH: Bayes-Optimal Meta-Learning with PAC-Guaranteescited by 85
  9. 2017Generalization Bounds of SGLD for Non-convex Learning: Two Theoretical Viewpointscited by 82
  10. 2017Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theorycited by 76
  11. 2020On the role of data in PAC-Bayes boundscited by 64
  12. 2020PAC-Bayes Analysis Beyond the Usual Boundscited by 63
24 more
  1. 2020Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networkscited by 62
  2. 2023Generalization Bounds: Perspectives from Information Theory and PAC-Bayescited by 60
  3. 2016Simpler PAC-Bayesian Bounds for Hostile Datacited by 45
  4. 2021Heavy Tails in SGD and Compressibility of Overparametrized Neural Networkscited by 44
  5. 2013A PAC-Bayesian Tutorial with A Dropout Boundcited by 43
  6. 2020PAC-Bayes unleashed: generalisation bounds with unbounded lossescited by 43
  7. 2021Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stabilitycited by 29
  8. 2021Generalization Bounds using Lower Tail Exponents in Stochastic Optimizerscited by 27
  9. 2023A unified recipe for deriving (time-uniform) PAC-Bayes boundscited by 25
  10. 2022Online PAC-Bayes Learningcited by 22
  11. 2021A General Framework for the Practical Disintegration of PAC-Bayesian Boundscited by 21
  12. 2021Bridging the Gap Between Practice and PAC-Bayes Theory in Few-Shot Meta-Learningcited by 21
  13. 2021Learning PAC-Bayes Priors for Probabilistic Neural Networkscited by 21
  14. 2019Fast-rate PAC-Bayes Generalization Bounds via Shifted Rademacher Processescited by 20
  15. 2022Integral Probability Metrics PAC-Bayes Boundscited by 19
  16. 2019Efron-Stein PAC-Bayesian Inequalitiescited by 15
  17. 2023Generalization Bounds with Data-dependent Fractal Dimensionscited by 15
  18. 2021A PAC-Bayes Analysis of Adversarial Robustnesscited by 13
  19. 2022Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squarescited by 12
  20. 2023Learning via Wasserstein-Based High Probability Generalisation Boundscited by 11
  21. 2022Change of measure through the Legendre transformcited by 10
  22. 2022Chaotic Regularization and Heavy-Tailed Limits for Deterministic Gradient Descentcited by 9
  23. 2023From Mutual Information to Expected Dynamics: New Generalization Bounds for Heavy-Tailed SGDcited by 4
  24. 2023Approximate Heavy Tails in Offline (Multi-Pass) Stochastic Gradient Descentcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024A PAC-Bayesian Link Between Generalisation and Flat Minimacited by 13
  2. 2024Generalization Bounds via Conditional f-Informationcited by 3