mlsanityData to 7 Oct 2026

PAC-Bayes with Backprop

arXiv 1908.07380 ↗First posted 19 Aug 2019cs.LG stat.MLcited by 36

Not on the map. Why?

Abstract (arXiv)

We explore the family of methods "PAC-Bayes with Backprop" (PBB) to train probabilistic neural networks by minimizing PAC-Bayes bounds. We present two training objectives, one derived from a previously known PAC-Bayes bound, and a second one derived from a novel PAC-Bayes bound. Both training objectives are evaluated on MNIST and on various UCI data sets. Our experiments show two striking observations: we obtain competitive test set error estimates (~1.4% on MNIST) and at the same time we compute non-vacuous bounds with much tighter values (~2.3% on MNIST) than previous results. These observations suggest that neural nets trained by PBB may lead to self-bounding learning, where the available data can be used to simultaneously learn a predictor and certify its risk, with no need to follow a data-splitting protocol.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 6; 2023: 12; 2024: 4; 2025: 3; 2026: 2.

36 papers.

Sort:

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  2. 2022PAC-Bayes Compression Bounds So Tight That They Can Explain Generalizationcited by 75
  3. 2022Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 42
  4. 2021Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stabilitycited by 29
  5. 2021How Tight Can PAC-Bayes be in the Small Data Regime?cited by 24
  6. 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 23
  7. 2022Online PAC-Bayes Learningcited by 22
  8. 2023Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regressioncited by 20
  9. 2023More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime validitycited by 17
  10. 2023PAC-tuning:Fine-tuning Pretrained Language Models with PAC-driven Perturbed Gradient Descentcited by 10
  11. 2024PAC-Bayes-Chernoff bounds for unbounded lossescited by 9
  12. 2023FedHB: Hierarchical Bayesian Federated Learningcited by 8
24 more
  1. 2022PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparisoncited by 7
  2. 2023Improving Generalization of Complex Models under Unbounded Loss Using PAC-Bayes Boundscited by 6
  3. 2023PAC-Bayes Generalization Certificates for Learned Inductive Conformal Predictioncited by 6
  4. 2021Task-Driven Detection of Distribution Shifts with Statistical Guarantees for Robot Learningcited by 4
  5. 2024Recursive PAC-Bayes: A Frequentist Approach to Sequential Prior Updates with No Information Losscited by 4
  6. 2021PAC Bayesian Performance Guarantees for Deep (Stochastic) Networks in Medical Imagingcited by 3
  7. 2023Wasserstein PAC-Bayes Learning: Exploiting Optimisation Guarantees to Explain Generalisationcited by 3
  8. 2023Federated Learning with Nonvacuous Generalisation Boundscited by 2
  9. 2024Towards Understanding Task-agnostic Debiasing Through the Lenses of Intrinsic Bias and Forgetfulnesscited by 2
  10. 2021Learning Provably Robust Motion Planners Using Funnel Librariescited by 1
  11. 2021Stronger Generalization Guarantees for Robot Learning by Combining Generative Models and Real-World Datacited by 1
  12. 2021PACMAN: PAC-style bounds accounting for the Mismatch between Accuracy and Negative log-losscited by 1
  13. 2024Learning via Surrogate PAC-Bayescited by 1
  14. 2025Some theoretical improvements on the tightness of PAC-Bayes risk certificates for neural networkscited by 1
  15. 2021Interactive Dynamic Walking: Learning Gait Switching Policies with Generalization Guaranteescited by 0
  16. 2022Streaming PAC-Bayes Gaussian process regression with a performance guarantee for online decision makingcited by 0
  17. 2023Fundamental Tradeoffs in Learning with Prior Informationcited by 0
  18. 2023A Hierarchical Bayesian Model for Deep Few-Shot Meta Learningcited by 0
  19. 2023EKGNet: A 10.96μW Fully Analog Neural Network for Intra-Patient Arrhythmia Classificationcited by 0
  20. 2023Optimization dependent generalization bound for ReLU networks based on sensitivity in the tangent bundlecited by 0
  21. 2025Deep Actor-Critics with Tight Risk Certificatescited by 0
  22. 2025How Patterns Dictate Learnability in Sequential Datacited by 0
  23. 2026From Privacy to Generalization: Linear Max-Information Bounds for Differentially Private Learning Algorithmscited by 0
  24. 2026Model-Agnostic Online Certificate-Driven Calibration for Time Series Forecasting Under Distribution Shiftcited by 0