mlsanityData to 7 Oct 2026

PAC-Bayes unleashed: generalisation bounds with unbounded losses

arXiv 2006.07279 ↗First posted 12 Jun 2020stat.ML cs.LG math.STcited by 43

Not on the map. Why?

Abstract (arXiv)

We present new PAC-Bayesian generalisation bounds for learning problems with unbounded loss functions. This extends the relevance and applicability of the PAC-Bayes learning framework, where most of the existing literature focuses on supervised learning problems with a bounded loss function (typically assumed to take values in the interval [0;1]). In order to relax this assumption, we propose a new notion called HYPE (standing for HYPothesis-dependent rangE), which effectively allows the range of the loss to depend on each predictor. Based on this new notion we derive a novel PAC-Bayesian generalisation bound for unbounded loss functions, and we instantiate it on a linear regression problem. To make our theory usable by the largest audience possible, we include discussions on actual computation, practicality and limitations of our assumptions.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 12; 2023: 11; 2024: 6; 2025: 3; 2026: 8.

43 papers.

Sort:

By area:

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  2. 2023Non-Vacuous Generalization Bounds for Large Language Modelscited by 53
  3. 2021Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stabilitycited by 29
  4. 2022A New Family of Generalization Bounds Using Samplewise Evaluated CMIcited by 28
  5. 2023A unified recipe for deriving (time-uniform) PAC-Bayes boundscited by 25
  6. 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 23
  7. 2022Online PAC-Bayes Learningcited by 22
  8. 2024Mission Impossible: A Statistical Perspective on Jailbreaking LLMscited by 21
  9. 2022On PAC-Bayesian reconstruction guarantees for VAEscited by 20
  10. 2023More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime validitycited by 17
  11. 2022Empirical Risk Minimization with Relative Entropy Regularizationcited by 13
  12. 2023PAC-Bayesian Generalization Bounds for Adversarial Generative Modelscited by 13
31 more
  1. 2021PAC-Bayes Bounds for Meta-learning with Data-Dependent Priorcited by 11
  2. 2023Learning via Wasserstein-Based High Probability Generalisation Boundscited by 11
  3. 2022Change of measure through the Legendre transformcited by 10
  4. 2023On the Validation of Gibbs Algorithms: Training Datasets, Test Datasets and their Aggregationcited by 10
  5. 2024PAC-Bayes-Chernoff bounds for unbounded lossescited by 9
  6. 2022Empirical Risk Minimization with Relative Entropy Regularization: Optimality and Sensitivity Analysiscited by 8
  7. 2022Shedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distancescited by 7
  8. 2023Generalization Analysis of Machine Learning Algorithms via the Worst-Case Data-Generating Probability Measurecited by 7
  9. 2022Weight Expansion: A New Perspective on Dropout and Generalizationcited by 6
  10. 2022On the Importance of Gradient Norm in PAC-Bayesian Boundscited by 6
  11. 2023Improving Generalization of Complex Models under Unbounded Loss Using PAC-Bayes Boundscited by 6
  12. 2023Comparing Comparators in Generalization Boundscited by 5
  13. 2024The Method of Gaps: Exact Expressions for the Generalization Error of Supervised Learning Algorithmscited by 5
  14. 2025Selective Risk Certification for LLM Outputs via Information-Lift Statistics: PAC-Bayes, Robustness, and Skeleton Designcited by 5
  15. 2024Sample Compression Unleashed: New Generalization Bounds for Real Valued Lossescited by 4
  16. 2023Wasserstein PAC-Bayes Learning: Exploiting Optimisation Guarantees to Explain Generalisationcited by 3
  17. 2025PAC-Bayes Bounds for Multivariate Linear Regression and Linear Autoencoderscited by 3
  18. 2024Tighter Generalisation Bounds via Interpolationcited by 2
  19. 2022Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learningcited by 1
  20. 2025Some theoretical improvements on the tightness of PAC-Bayes risk certificates for neural networkscited by 1
  21. 2026Smoothness-Based Derandomization of PAC-Bayes Boundscited by 1
  22. 2026PAC-Bayesian Reconstruction Guarantees for Time Series Variational Autoencoderscited by 1
  23. 2022Streaming PAC-Bayes Gaussian process regression with a performance guarantee for online decision makingcited by 0
  24. 2023On Certified Generalization in Structured Predictioncited by 0
  25. 2024A note on generalization bounds for losses with finite momentscited by 0
  26. 2026Empirical Risk Minimization with f-Divergence Regularizationcited by 0
  27. 2026Generalization Bounds of Stochastic Gradient Descent in Homogeneous Neural Networkscited by 0
  28. 2026General Bayesian Policy Learningcited by 0
  29. 2026A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learningcited by 0
  30. 2026Conservative neural posterior estimation via distributionally robust trainingcited by 0
  31. 2026How Far Can Sharpness and Complexity Jointly Explain Generalization?cited by 0