mlsanityData to 14 Oct 2026

A Strongly Quasiconvex PAC-Bayesian Bound

arXiv 1608.05610 ↗First posted 19 Aug 2016cs.LG stat.MLcited by 54

Not on the map. Why?

Abstract (arXiv)

We propose a new PAC-Bayesian bound and a way of constructing a hypothesis space, so that the bound is convex in the posterior distribution and also convex in a trade-off parameter between empirical performance of the posterior distribution and its complexity. The complexity is measured by the Kullback-Leibler divergence to a prior. We derive an alternating procedure for minimizing the bound. We show that the bound can be rewritten as a one-dimensional function of the trade-off parameter and provide sufficient conditions under which the function has a single global minimum. When the conditions are satisfied the alternating minimization is guaranteed to converge to the global minimum of the bound. We provide experimental results demonstrating that rigorous minimization of the bound is competitive with cross-validation in tuning the trade-off between complexity and empirical performance. In all our experiments the trade-off turned to be quasiconvex even when the sufficient conditions were violated.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 13; 2023: 12; 2024: 7; 2025: 7; 2026: 2.

54 papers.

Sort:

By area:

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  2. 2022PAC-Bayes Compression Bounds So Tight That They Can Explain Generalizationcited by 76
  3. 2022Bayesian Model Selection, the Marginal Likelihood, and Generalizationcited by 57
  4. 2022Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 42
  5. 2021Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stabilitycited by 30
  6. 2023A unified recipe for deriving (time-uniform) PAC-Bayes boundscited by 25
  7. 2021How Tight Can PAC-Bayes be in the Small Data Regime?cited by 24
  8. 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 24
  9. 2022Online PAC-Bayes Learningcited by 22
  10. 2023Sparse is Enough in Fine-tuning Pre-trained Large Language Modelscited by 21
  11. 2023More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime validitycited by 17
  12. 2021Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Boundcited by 16
38 more
  1. 2023Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theorycited by 16
  2. 2021Information Complexity and Generalization Boundscited by 14
  3. 2021Chebyshev-Cantelli PAC-Bayes-Bennett Inequality for the Weighted Majority Votecited by 13
  4. 2021Information-Theoretic Bounds on the Moments of the Generalization Error of Learning Algorithmscited by 12
  5. 2021PAC-Bayes Bounds for Meta-learning with Data-Dependent Priorcited by 11
  6. 2022Split-kl and PAC-Bayes-split-kl Inequalities for Ternary Random Variablescited by 11
  7. 2021Wide stochastic networks: Gaussian limit and PAC-Bayesian trainingcited by 10
  8. 2023Data-dependent Generalization Bounds via Variable-Size Compressibilitycited by 9
  9. 2024PAC-Bayes-Chernoff bounds for unbounded lossescited by 9
  10. 2023FedHB: Hierarchical Bayesian Federated Learningcited by 8
  11. 2021Conditionally Gaussian PAC-Bayescited by 7
  12. 2022On Margins and Generalisation for Voting Classifierscited by 7
  13. 2022PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparisoncited by 7
  14. 2022Weight Expansion: A New Perspective on Dropout and Generalizationcited by 6
  15. 2023Improving Generalization of Complex Models under Unbounded Loss Using PAC-Bayes Boundscited by 6
  16. 2021Task-Driven Detection of Distribution Shifts with Statistical Guarantees for Robot Learningcited by 4
  17. 2022PAC-Bayesian Learning of Optimization Algorithmscited by 4
  18. 2024Recursive PAC-Bayes: A Frequentist Approach to Sequential Prior Updates with No Information Losscited by 4
  19. 2021Multi-class Probabilistic Bounds for Self-learningcited by 3
  20. 2022A General framework for PAC-Bayes Bounds for Meta-Learningcited by 3
  21. 2024Generalization Bounds via Meta-Learned Model Representations: PAC-Bayes and Sample Compression Hypernetworkscited by 3
  22. 2025A Framework for Bounding Deterministic Risk with PAC-Bayes: Applications to Majority Votescited by 3
  23. 2022Self-Certifying Classification by Linearized Deep Assignmentcited by 2
  24. 2024Geometric Insights into Focal Loss: Reducing Curvature for Enhanced Model Calibrationcited by 2
  25. 2025Generalization Guarantees for Representation Learning via Data-Dependent Gaussian Mixture Priorscited by 2
  26. 2025Reconcile Certified Robustness and Accuracy for DNN-based Smoothed Majority Vote Classifiercited by 2
  27. 2021Stronger Generalization Guarantees for Robot Learning by Combining Generative Models and Real-World Datacited by 1
  28. 2022Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learningcited by 1
  29. 2023Mixed moving average field guided learning for spatio-temporal datacited by 1
  30. 2023Sequential Gibbs Posteriors with Applications to Principal Component Analysiscited by 1
  31. 2023Estimating optimal PAC-Bayes bounds with Hamiltonian Monte Carlocited by 1
  32. 2024A Generalization Result for Convergence in Learning-to-Optimizecited by 1
  33. 2024Multi-View Majority Vote Learning Algorithms: Direct Minimization of PAC-Bayesian Boundscited by 1
  34. 2025Multiview Representation Learning via Distributed Joint Latent Space Structuringcited by 1
  35. 2025Some theoretical improvements on the tightness of PAC-Bayes risk certificates for neural networkscited by 1
  36. 2025PAC-Bayesian Reinforcement Learning Trains Generalizable Policiescited by 1
  37. 2023A Hierarchical Bayesian Model for Deep Few-Shot Meta Learningcited by 0
  38. 2023Something for (almost) nothing: Improving deep ensemble calibration using unlabeled datacited by 0

50 most cited of 54 All 54 (JSON)