mlsanityData to 7 Oct 2026

PAC-Bayes under potentially heavy tails

arXiv 1905.07900 ↗First posted 20 May 2019stat.ML cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

We derive PAC-Bayesian learning guarantees for heavy-tailed losses, and obtain a novel optimal Gibbs posterior which enjoys finite-sample excess risk bounds at logarithmic confidence. Our core technique itself makes use of PAC-Bayesian inequalities in order to derive a robust risk estimator, which by design is easy to compute. In particular, only assuming that the first three moments of the loss distribution are bounded, the learning algorithm derived from this estimator achieves nearly sub-Gaussian statistical error, up to the quality of the prior.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 5; 2023: 5; 2024: 3; 2025: 0; 2026: 1.

16 papers.

Sort:

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  2. 2023Non-Vacuous Generalization Bounds for Large Language Modelscited by 53
  3. 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 23
  4. 2022Online PAC-Bayes Learningcited by 22
  5. 2022On PAC-Bayesian reconstruction guarantees for VAEscited by 20
  6. 2023More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime validitycited by 17
  7. 2021PAC-Bayes Bounds for Meta-learning with Data-Dependent Priorcited by 11
  8. 2022Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practicecited by 11
  9. 2024PAC-Bayes-Chernoff bounds for unbounded lossescited by 9
  10. 2022On the Importance of Gradient Norm in PAC-Bayesian Boundscited by 6
  11. 2023Improving Generalization of Complex Models under Unbounded Loss Using PAC-Bayes Boundscited by 6
  12. 2024Generalization and Robustness of the Tilted Empirical Riskcited by 4
4 more
  1. 2023Wasserstein PAC-Bayes Learning: Exploiting Optimisation Guarantees to Explain Generalisationcited by 3
  2. 2023Mixed moving average field guided learning for spatio-temporal datacited by 1
  3. 2024A note on generalization bounds for losses with finite momentscited by 0
  4. 2026A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learningcited by 0