mlsanityData to 14 Oct 2026

Conditionally Gaussian PAC-Bayes

arXiv 2110.11886 ↗First posted 22 Oct 2021cs.LG stat.MLcited by 7

Not on the map. Why?

Abstract (arXiv)

Recent studies have empirically investigated different methods to train stochastic neural networks on a classification task by optimising a PAC-Bayesian bound via stochastic gradient descent. Most of these procedures need to replace the misclassification error with a surrogate loss, leading to a mismatch between the optimisation objective and the actual generalisation bound. The present paper proposes a novel training algorithm that optimises the PAC-Bayesian bound, without relying on any surrogate loss. Empirical results show that this approach outperforms currently available PAC-Bayesian training methods.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 20 of its 40 references that are in the map. First posted on arXiv 22 Oct 2021.

Builds on

20 of 40 references.

References · 20

  1. 2017Exploring Generalization in Deep Learningcited by 600
  2. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  3. 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
  4. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  5. 2020Better Theory for SGD in the Nonconvex Worldcited by 158
  6. 2017Generalization in Deep Learningcited by 146
  7. 2019A Primer on PAC-Bayesian Learningcited by 132
  8. 2018Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approachcited by 126
  9. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
  10. 2020Tighter risk certificates for neural networkscited by 112
  11. 2017Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priorscited by 81
  12. 2020Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networkscited by 73
8 more
  1. 2019Theoretical Issues in Deep Networks: Approximation, Optimization and Generalizationcited by 72
  2. 2020On the role of data in PAC-Bayes boundscited by 64
  3. 2016A Strongly Quasiconvex PAC-Bayesian Boundcited by 54
  4. 2019Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networkscited by 38
  5. 2020Differentiable PAC-Bayes Objectives with Partially Aggregated Neural Networkscited by 29
  6. 2021Learning PAC-Bayes Priors for Probabilistic Neural Networkscited by 21
  7. 2017Asymptotic Bias of Stochastic Gradient Searchcited by 19
  8. 2021Wide stochastic networks: Gaussian limit and PAC-Bayesian trainingcited by 10

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 2; 2024: 0; 2025: 1.

7 papers.

  1. 2022Generalisation under gradient descent via deterministic PAC-Bayescited by 9
  2. 2025How good is PAC-Bayes at explaining generalisation?cited by 4
  3. 2022Controlling Multiple Errors Simultaneously with a PAC-Bayes Boundcited by 3
  4. 2022A Note on the Efficient Evaluation of PAC-Bayes Boundscited by 2
  5. 2023A note on regularised NTK dynamics with an application to PAC-Bayesian trainingcited by 1
  6. 2022Streaming PAC-Bayes Gaussian process regression with a performance guarantee for online decision makingcited by 0
  7. 2023On Certified Generalization in Structured Predictioncited by 0