mlsanityData to 14 Oct 2026

Online PAC-Bayes Learning

arXiv 2206.00024 ↗First posted 31 May 2022cs.LG math.ST stat.MLcited by 22

Not on the map. Why?

Abstract (arXiv)

Most PAC-Bayesian bounds hold in the batch learning setting where data is collected at once, prior to inference or prediction. This somewhat departs from many contemporary learning problems where data streams are collected and the algorithms must dynamically adjust. We prove new PAC-Bayesian bounds in this online learning framework, leveraging an updated definition of regret, and we revisit classical PAC-Bayesian results with a batch-to-online conversion, extending their remit to the case of dependent data. Our results hold for bounded losses, potentially non-convex, paving the way to promising developments in online learning.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 27 of its 66 references that are in the map. First posted on arXiv 31 May 2022.

Builds on

27 of 66 references.

References · 27

  1. 2019Introduction to Online Convex Optimizationcited by 803
  2. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  3. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  4. 2013Optimization, Learning, and Games with Predictable Sequencescited by 188
  5. 2015Adding Gradient Noise Improves Learning for Very Deep Networkscited by 145
  6. 2019A Primer on PAC-Bayesian Learningcited by 132
  7. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
  8. 2020Tighter risk certificates for neural networkscited by 112
  9. 2020Dynamic Regret of Convex and Smooth Functionscited by 90
  10. 2016PAC-Bayesian Theory Meets Bayesian Inferencecited by 88
  11. 2016Tracking Slowly Moving Clairvoyant: Optimal Dynamic Regret of Online Learning with True and Noisy Gradientcited by 66
  12. 2020PAC-Bayes Analysis Beyond the Usual Boundscited by 63
15 more
  1. 2016A Strongly Quasiconvex PAC-Bayesian Boundcited by 54
  2. 2016Simpler PAC-Bayesian Bounds for Hostile Datacited by 46
  3. 2020PAC-Bayes unleashed: generalisation bounds with unbounded lossescited by 43
  4. 2019Spiking Neural Networks and Online Learning: An Overview and Perspectivescited by 41
  5. 2019PAC-Bayes Un-Expected Bernstein Inequalitycited by 36
  6. 2019PAC-Bayes with Backpropcited by 36
  7. 2018The Many Faces of Exponential Weights in Online Learningcited by 33
  8. 2022Generalization Bounds via Convex Analysiscited by 27
  9. 2015Adaptive Online Learningcited by 23
  10. 2021Learning PAC-Bayes Priors for Probabilistic Neural Networkscited by 21
  11. 2019A Generalization Bound for Online Variational Inferencecited by 17
  12. 2019PAC-Bayes under potentially heavy tailscited by 16
  13. 2019Still no free lunches: the price to pay for tighter PAC-Bayes boundscited by 14
  14. 2021Progress in Self-Certified Neural Networkscited by 10
  15. 2021Stochastic Online Convex Optimization. Application to probabilistic time series forecastingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 8; 2024: 5; 2025: 2; 2026: 5.

22 papers.

Sort:

  1. 2023MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splittingcited by 33
  2. 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 24
  3. 2023Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theorycited by 16
  4. 2023PAC-Bayesian Generalization Bounds for Adversarial Generative Modelscited by 13
  5. 2024A PAC-Bayesian Link Between Generalisation and Flat Minimacited by 13
  6. 2023Learning via Wasserstein-Based High Probability Generalisation Boundscited by 12
  7. 2023Improved Algorithms for Stochastic Linear Bandits Using Tail Bounds for Martingale Mixturescited by 12
  8. 2024Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learningcited by 12
  9. 2023Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often!cited by 8
  10. 2023Wasserstein PAC-Bayes Learning: Exploiting Optimisation Guarantees to Explain Generalisationcited by 3
  11. 2024Data-dependent and Oracle Bounds on Forgetting in Continual Learningcited by 3
  12. 2023Federated Learning with Nonvacuous Generalisation Boundscited by 2
10 more
  1. 2024Tighter Generalisation Bounds via Interpolationcited by 2
  2. 2024Length independent generalization bounds for deep SSM architectures via Rademacher contraction and stability constraintscited by 2
  3. 2025Reconcile Certified Robustness and Accuracy for DNN-based Smoothed Majority Vote Classifiercited by 2
  4. 2026Adaptive Bayesian Online Learning via Expert Aggregationcited by 1
  5. 2022Streaming PAC-Bayes Gaussian process regression with a performance guarantee for online decision makingcited by 0
  6. 2025Sequential Off-Policy Learning with Logarithmic Smoothingcited by 0
  7. 2026Block-Sample MAC-Bayes Generalization Boundscited by 0
  8. 2026Personalized Federated Learning for Gradient Alignmentcited by 0
  9. 2026Information-Theoretic Generalization Bounds for Sequential Decision Makingcited by 0
  10. 2026Certified Policy Optimisation for Nested Causal Bandits via PAC-Bayes Riskcited by 0