mlsanityData to 7 Oct 2026

Differentiable PAC-Bayes Objectives with Partially Aggregated Neural Networks

arXiv 2006.12228 ↗First posted 22 Jun 2020cs.LG stat.MLcited by 29

Not on the map. Why?

Abstract (arXiv)

We make three related contributions motivated by the challenge of training stochastic neural networks, particularly in a PAC-Bayesian setting: (1) we show how averaging over an ensemble of stochastic neural networks enables a new class of partially-aggregated estimators; (2) we show that these lead to provably lower-variance gradient estimates for non-differentiable signed-output networks; (3) we reformulate a PAC-Bayesian bound for these networks to derive a directly optimisable, differentiable objective and a generalisation guarantee, without using a surrogate loss or loosening the bound. This bound is twice as tight as that of Letarte et al. (2019) on a similar network type. We show empirically that these innovations make training easier and lead to competitive guarantees.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 8; 2023: 9; 2024: 1; 2025: 2; 2026: 1.

29 papers.

Sort:

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  2. 2023MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splittingcited by 33
  3. 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 23
  4. 2022Non-Vacuous Generalisation Bounds for Shallow Neural Networkscited by 22
  5. 2021A General Framework for the Practical Disintegration of PAC-Bayesian Boundscited by 21
  6. 2021Learning PAC-Bayes Priors for Probabilistic Neural Networkscited by 21
  7. 2022On PAC-Bayesian reconstruction guarantees for VAEscited by 20
  8. 2021On Margins and Derandomisation in PAC-Bayescited by 19
  9. 2022On Rademacher Complexity-based Generalization Bounds for Deep Learningcited by 16
  10. 2023Learning via Wasserstein-Based High Probability Generalisation Boundscited by 11
  11. 2021Wide stochastic networks: Gaussian limit and PAC-Bayesian trainingcited by 10
  12. 2021Generalization Error Bounds on Deep Learning with Markov Datasetscited by 9
17 more
  1. 2022Generalisation under gradient descent via deterministic PAC-Bayescited by 9
  2. 2022Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficultycited by 9
  3. 2021Conditionally Gaussian PAC-Bayescited by 7
  4. 2022On Margins and Generalisation for Voting Classifierscited by 7
  5. 2023Improving Generalization of Complex Models under Unbounded Loss Using PAC-Bayes Boundscited by 6
  6. 2023Comparing Comparators in Generalization Boundscited by 5
  7. 2023Global Convergence Rate of Deep Equilibrium Models with General Activationscited by 4
  8. 2023Wasserstein PAC-Bayes Learning: Exploiting Optimisation Guarantees to Explain Generalisationcited by 3
  9. 2025A Framework for Bounding Deterministic Risk with PAC-Bayes: Applications to Majority Votescited by 3
  10. 2023Federated Learning with Nonvacuous Generalisation Boundscited by 2
  11. 2021PAC-Bayesian Learning of Aggregated Binary Activated Neural Networks with Probabilities over Representationscited by 1
  12. 2022Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learningcited by 1
  13. 2023A note on regularised NTK dynamics with an application to PAC-Bayesian trainingcited by 1
  14. 2024Learning via Surrogate PAC-Bayescited by 1
  15. 2025Non-Vacuous Generalization Bounds: Can Rescaling Invariances Help?cited by 1
  16. 2026PAC-Bayesian Reconstruction Guarantees for Time Series Variational Autoencoderscited by 1
  17. 2023The surrogate Gibbs-posterior of a corrected stochastic MALA: Towards uncertainty quantification for neural networkscited by 0