mlsanityData to 7 Oct 2026

Simpler PAC-Bayesian Bounds for Hostile Data

arXiv 1610.07193 ↗First posted 23 Oct 2016stat.ML math.ST stat.THcited by 45

Not on the map. Why?

Abstract (arXiv)

PAC-Bayesian learning bounds are of the utmost interest to the learning community. Their role is to connect the generalization ability of an aggregation distribution ρ to its empirical risk and to its Kullback-Leibler divergence with respect to some prior distribution π. Unfortunately, most of the available bounds typically rely on heavy assumptions such as boundedness and independence of the observations. This paper aims at relaxing these constraints and provides PAC-Bayesian learning bounds that hold for dependent, heavy-tailed observations (hereafter referred to as hostile data). In these bounds the Kullack-Leibler divergence is replaced with a general version of Csisz'ar's f-divergence. We prove a general PAC-Bayesian bound, and show how to use it in various hostile settings.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Oct 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 12; 2023: 11; 2024: 8; 2025: 6; 2026: 4.

45 papers.

Sort:

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  2. 2023A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methodscited by 31
  3. 2022Generalization Bounds via Convex Analysiscited by 27
  4. 2023A unified recipe for deriving (time-uniform) PAC-Bayes boundscited by 25
  5. 2021How Tight Can PAC-Bayes be in the Small Data Regime?cited by 24
  6. 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 23
  7. 2022Online PAC-Bayes Learningcited by 22
  8. 2022On PAC-Bayesian reconstruction guarantees for VAEscited by 20
  9. 2024Position: Understanding LLMs Requires More Than Statistical Generalizationcited by 20
  10. 2022Integral Probability Metrics PAC-Bayes Boundscited by 19
  11. 2023More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime validitycited by 17
  12. 2023Online-to-PAC Conversions: Generalization Bounds via Regret Analysiscited by 14
33 more
  1. 2022Instance-Dependent Generalization Bounds via Optimal Transportcited by 13
  2. 2021Information-Theoretic Bounds on the Moments of the Generalization Error of Learning Algorithmscited by 12
  3. 2022Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practicecited by 11
  4. 2023Learning via Wasserstein-Based High Probability Generalisation Boundscited by 11
  5. 2022Change of measure through the Legendre transformcited by 10
  6. 2024PAC-Bayes-Chernoff bounds for unbounded lossescited by 9
  7. 2022Shedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distancescited by 7
  8. 2021Stopping Criterion for Active Learning Based on Error Stabilitycited by 6
  9. 2022On the Importance of Gradient Norm in PAC-Bayesian Boundscited by 6
  10. 2023Improving Generalization of Complex Models under Unbounded Loss Using PAC-Bayes Boundscited by 6
  11. 2024Tight PAC-Bayesian Risk Certificates for Contrastive Learningcited by 6
  12. 2023Comparing Comparators in Generalization Boundscited by 5
  13. 2022PAC-Bayesian Learning of Optimization Algorithmscited by 4
  14. 2024Better-than-KL PAC-Bayes Boundscited by 4
  15. 2024Generalization and Robustness of the Tilted Empirical Riskcited by 4
  16. 2025How good is PAC-Bayes at explaining generalisation?cited by 4
  17. 2023PAC-Bayesian bounds for learning LTI-ss systems with input from empirical losscited by 3
  18. 2023Wasserstein PAC-Bayes Learning: Exploiting Optimisation Guarantees to Explain Generalisationcited by 3
  19. 2024Tighter Generalisation Bounds via Interpolationcited by 2
  20. 2025Empirical PAC-Bayes Bounds for Markov Chainscited by 2
  21. 2022PAC-Bayesian-Like Error Bound for a Class of Linear Time-Invariant Stochastic State-Space Modelscited by 1
  22. 2023Mixed moving average field guided learning for spatio-temporal datacited by 1
  23. 2024A Generalization Result for Convergence in Learning-to-Optimizecited by 1
  24. 2025Make Optimization Once and for All with Fine-grained Guidancecited by 1
  25. 2025Non-Vacuous Generalization Bounds: Can Rescaling Invariances Help?cited by 1
  26. 2026PAC-Bayesian Reconstruction Guarantees for Time Series Variational Autoencoderscited by 1
  27. 2023On Certified Generalization in Structured Predictioncited by 0
  28. 2024A note on generalization bounds for losses with finite momentscited by 0
  29. 2025Effective Sample Size and Generalization Bounds for Temporal Networkscited by 0
  30. 2025PAC-Bayesian Generalization Bounds for Graph Convolutional Networks on Inductive Node Classificationcited by 0
  31. 2026Block-Sample MAC-Bayes Generalization Boundscited by 0
  32. 2026Spatio-temporal probabilistic forecast using MMAF-guided learningcited by 0
  33. 2026A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learningcited by 0