mlsanityData to 7 Oct 2026

Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures

arXiv 2402.13285 ↗First posted 19 Feb 2024stat.ML cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

In statistical learning theory, a generalization bound usually involves a complexity measure imposed by the considered theoretical framework. This limits the scope of such bounds, as other forms of capacity measures or regularizations are used in algorithms. In this paper, we leverage the framework of disintegrated PAC-Bayes bounds to derive a general generalization bound instantiable with arbitrary complexity measures. One trick to prove such a result involves considering a commonly used family of distributions: the Gibbs distributions. Our bound stands in probability jointly over the hypothesis and the learning sample, which allows the complexity to be adapted to the generalization gap as it can be customized to fit both the hypothesis class and the task.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 28 of its 64 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

28 of 64 references.

References · 28

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  4. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  5. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  6. 2017Sharp Minima Can Generalize For Deep Netscited by 462
  7. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 299
  8. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 242
  9. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  10. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 223
  11. 2019A Primer on PAC-Bayesian Learningcited by 130
  12. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
16 more
  1. 2015Path-SGD: Path-Normalized Optimization in Deep Neural Networkscited by 117
  2. 2019Tightening Mutual Information Based Bounds on Generalization Errorcited by 117
  3. 2018Predicting the Generalization Gap in Deep Networks with Margin Distributionscited by 111
  4. 2017Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priorscited by 81
  5. 2020In Search of Robust Measures of Generalizationcited by 79
  6. 2015Generalization in Adaptive Data Analysis and Holdout Reusecited by 76
  7. 2020On the role of data in PAC-Bayes boundscited by 64
  8. 2020PAC-Bayes Analysis Beyond the Usual Boundscited by 63
  9. 2023Generalization Bounds: Perspectives from Information Theory and PAC-Bayescited by 60
  10. 2020Generalization Bounds via Information Density and Conditional Information Densitycited by 43
  11. 2019Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networkscited by 38
  12. 2018Learning Gaussian Processes by Minimizing PAC-Bayesian Generalization Boundscited by 24
  13. 2021A General Framework for the Practical Disintegration of PAC-Bayesian Boundscited by 21
  14. 2019Distribution-Dependent Analysis of Gibbs-ERM Principlecited by 17
  15. 2020Neural Complexity Measurescited by 11
  16. 2016PAC-Bayesian Analysis for a two-step Hierarchical Multiview Learning Approachcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 4.

7 papers.

  1. 2024Which Algorithms Have Tight Generalization Bounds?cited by 1
  2. 2025PAC-Bayesian Bounds on Constrained f-Entropic Risk Measurescited by 1
  3. 2026PAC-Bayes Beyond Parameter Space: Behavioral Equivalence, Z-Information, and Exact Complexity Decompositioncited by 1
  4. 2025Generalization of the Gibbs algorithm with high probability at low temperaturescited by 0
  5. 2026PAC-Bayesian Generalization Guarantees for Fairness on Stochastic and Deterministic Classifierscited by 0
  6. 2026Certification of Machine Learning Models via Directional Sharpnesscited by 0
  7. 2026Certification Frontiers for Gaussian LoRA: Independent Priors, Posterior Risk, and Prediction-Preserving Balancingcited by 0