mlsanityData to 14 Oct 2026

PAC-Bayesian Theory Meets Bayesian Inference

arXiv 1605.08636 ↗First posted 27 May 2016stat.ML cs.LGcited by 88

Not on the map. Why?

Abstract (arXiv)

We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanation to the Bayesian Occam's razor criteria, under the assumption that the data is generated by an i.i.d distribution. Moreover, as the negative log-likelihood is an unbounded loss function, we motivate and propose a PAC-Bayesian theorem tailored for the sub-gamma loss family, and we show that our approach is sound on classical Bayesian linear regression tasks.

Key facts. Cited by 88 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 20; 2023: 20; 2024: 5; 2025: 9; 2026: 18.

88 papers.

Sort:

By area:

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  2. 2021Bayesian Neural Network Priors Revisitedcited by 121
  3. 2023The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learningcited by 63
  4. 2022Bayesian Model Selection, the Marginal Likelihood, and Generalizationcited by 57
  5. 2025Deep Learning is Not So Mysterious or Differentcited by 56
  6. 2023Non-Vacuous Generalization Bounds for Large Language Modelscited by 53
  7. 2021Generalization Bounds For Meta-Learning: An Information-Theoretic Analysiscited by 45
  8. 2022Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximationscited by 42
  9. 2021Diversity and Generalization in Neural Network Ensemblescited by 32
  10. 2022PACTran: PAC-Bayesian Metrics for Estimating the Transferability of Pretrained Models to Classification Taskscited by 30
  11. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 27
  12. 2021How Tight Can PAC-Bayes be in the Small Data Regime?cited by 24
38 more
  1. 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 24
  2. 2022Online PAC-Bayes Learningcited by 22
  3. 2021Bridging the Gap Between Practice and PAC-Bayes Theory in Few-Shot Meta-Learningcited by 21
  4. 2023Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theorycited by 16
  5. 2021Information Complexity and Generalization Boundscited by 14
  6. 2021Learning Augmentation Distributions using Transformed Risk Minimizationcited by 13
  7. 2023PAC-Bayesian Generalization Bounds for Adversarial Generative Modelscited by 13
  8. 2024Which Model to Transfer? A Survey on Transferability Estimationcited by 13
  9. 2023The Interpolating Information Criterion for Overparameterized Modelscited by 12
  10. 2021PAC-Bayes Bounds for Meta-learning with Data-Dependent Priorcited by 11
  11. 2022Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processescited by 11
  12. 2022Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practicecited by 11
  13. 2025Models of Heavy-Tailed Mechanistic Universalitycited by 10
  14. 2024PAC-Bayes-Chernoff bounds for unbounded lossescited by 9
  15. 2024Bridging SFT and DPO for Diffusion Model Alignment with Self-Sampling Preference Optimizationcited by 9
  16. 2021Weak Adaptation Learning -- Addressing Cross-domain Data Insufficiency with Weak Annotatorcited by 8
  17. 2023Hyperparameter Optimization through Neural Network Partitioningcited by 8
  18. 2021PAC-Bayesian theory for stochastic LTI systemscited by 7
  19. 2021Stopping Criterion for Active Learning Based on Error Stabilitycited by 6
  20. 2021On out-of-distribution detection with Bayesian neural networkscited by 6
  21. 2022On the Importance of Calibration in Semi-supervised Learningcited by 6
  22. 2022On the Importance of Gradient Norm in PAC-Bayesian Boundscited by 6
  23. 2022Improving Robust Generalization by Direct PAC-Bayesian Bound Minimizationcited by 6
  24. 2022Disentangling Model Multiplicity in Deep Learningcited by 5
  25. 2022Revisiting Active Sets for Gaussian Process Decoderscited by 5
  26. 2023Cheap and Deterministic Inference for Deep State-Space Models of Interacting Dynamical Systemscited by 5
  27. 2023Comparing Comparators in Generalization Boundscited by 5
  28. 2023A PAC-Bayesian Perspective on the Interpolating Information Criterioncited by 5
  29. 2023Learning Against Distributional Uncertainty: On the Trade-off Between Robustness and Specificitycited by 4
  30. 2023PAC-Bayesian bounds for learning LTI-ss systems with input from empirical losscited by 4
  31. 2023Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networkscited by 4
  32. 2023PAC-Bayes Generalisation Bounds for Dynamical Systems Including Stable RNNscited by 4
  33. 2026Towards A Unified PAC-Bayesian Framework for Norm-based Generalization Boundscited by 4
  34. 2021Loss function based second-order Jensen inequality and its application to particle variational inferencecited by 3
  35. 2022Cold Posteriors through PAC-Bayescited by 3
  36. 2023The fine print on tempered posteriorscited by 3
  37. 2024Bayes without Underfitting: Fully Correlated Deep Learning Posteriors via Alternating Projectionscited by 3
  38. 2025PAC-Bayes Bounds for Multivariate Linear Regression and Linear Autoencoderscited by 3

50 most cited of 88 All 88 (JSON)