mlsanityData to 7 Oct 2026

How good is PAC-Bayes at explaining generalisation?

arXiv 2503.08231 ↗First posted 11 Mar 2025stat.ML cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We discuss necessary conditions for a PAC-Bayes bound to provide a meaningful generalisation guarantee. Our analysis reveals that the optimal generalisation guarantee depends solely on the distribution of the risk induced by the prior distribution. In particular, achieving a target generalisation level is only achievable if the prior places sufficient mass on high-performing predictors. We relate these requirements to the prevalent practice of using data-dependent priors in deep learning PAC-Bayes applications, and discuss the implications for the claim that PAC-Bayes ``explains'' generalisation.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 16 of its 31 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

16 of 31 references.

References · 16

  1. 2017Deep Neural Networks as Gaussian Processescited by 617
  2. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  3. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  4. 2016Deep Information Propagationcited by 194
  5. 2019A Primer on PAC-Bayesian Learningcited by 130
  6. 2018Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approachcited by 125
  7. 2020Tighter risk certificates for neural networkscited by 112
  8. 2019On the Impact of the Activation Function on Deep Neural Networks Trainingcited by 101
  9. 2022PAC-Bayes Compression Bounds So Tight That They Can Explain Generalizationcited by 75
  10. 2020On the role of data in PAC-Bayes boundscited by 64
  11. 2016Simpler PAC-Bayesian Bounds for Hostile Datacited by 45
  12. 2019PAC-Bayes Un-Expected Bernstein Inequalitycited by 36
4 more
  1. 2021Learning PAC-Bayes Priors for Probabilistic Neural Networkscited by 21
  2. 2023Tighter PAC-Bayes Bounds Through Coin-Bettingcited by 13
  3. 2023Learning via Wasserstein-Based High Probability Generalisation Boundscited by 11
  4. 2021Conditionally Gaussian PAC-Bayescited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025The Alignment Bottleneckcited by 1
  2. 2026Empirical Risk Minimization with f-Divergence Regularizationcited by 0
  3. 2026Can Circuit Alignment Predict OOD Generalization?cited by 0
  4. 2026Certification Frontiers for Gaussian LoRA: Independent Priors, Posterior Risk, and Prediction-Preserving Balancingcited by 0