mlsanityData to 7 Oct 2026

A PAC-Bayesian Link Between Generalisation and Flat Minima

arXiv 2402.08508 ↗First posted 13 Feb 2024stat.ML cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Modern machine learning usually involves predictors in the overparameterised setting (number of trained parameters greater than dataset size), and their training yields not only good performance on training data, but also good generalisation capacity. This phenomenon challenges many theoretical results, and remains an open problem. To reach a better understanding, we provide novel generalisation bounds involving gradient terms. To do so, we combine the PAC-Bayes toolbox with Poincar'e and Log-Sobolev inequalities, avoiding an explicit dependency on the dimension of the predictor space. Our results highlight the positive influence of flat minima (being minima with a neighbourhood nearly minimising the learning problem as well) on generalisation performance, involving directly the benefits of the optimisation phase.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 39 of its 65 references that are in the map. First posted on arXiv 13 Feb 2024.

Builds on

39 of 65 references.

References · 39

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  4. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  5. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  6. 2017Exploring Generalization in Deep Learningcited by 595
  7. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  8. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  9. 2017Three Factors Influencing Minima in SGDcited by 270
  10. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  11. 2023Handbook of Convergence Theorems for (Stochastic) Gradient Methodscited by 158
  12. 2019A Primer on PAC-Bayesian Learningcited by 130
27 more
  1. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
  2. 2019Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimatescited by 118
  3. 2023A Modern Look at the Relationship between Sharpness and Generalizationcited by 91
  4. 2020Relative Flatness and Generalizationcited by 88
  5. 2019Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networkscited by 86
  6. 2021Information-Theoretic Generalization Bounds for Stochastic Gradient Descentcited by 84
  7. 2020In Search of Robust Measures of Generalizationcited by 79
  8. 2020PAC-Bayes Analysis Beyond the Usual Boundscited by 63
  9. 2023Generalization Bounds: Perspectives from Information Theory and PAC-Bayescited by 60
  10. 2023Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalizationcited by 39
  11. 2019PAC-Bayes Un-Expected Bernstein Inequalitycited by 36
  12. 2016Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regionscited by 33
  13. 2021PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classescited by 31
  14. 2022Generalization Bounds via Convex Analysiscited by 27
  15. 2023A unified recipe for deriving (time-uniform) PAC-Bayes boundscited by 25
  16. 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 23
  17. 2022Online PAC-Bayes Learningcited by 22
  18. 2021A General Framework for the Practical Disintegration of PAC-Bayesian Boundscited by 21
  19. 2022Integral Probability Metrics PAC-Bayes Boundscited by 19
  20. 2023Fantastic Generalization Measures are Nowhere to be Foundcited by 19
  21. 2019Distribution-Dependent Analysis of Gibbs-ERM Principlecited by 17
  22. 2023Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Termcited by 16
  23. 2023Online-to-PAC Conversions: Generalization Bounds via Regret Analysiscited by 14
  24. 2023Learning via Wasserstein-Based High Probability Generalisation Boundscited by 11
  25. 2021Progress in Self-Certified Neural Networkscited by 10
  26. 2022On the Importance of Gradient Norm in PAC-Bayesian Boundscited by 6
  27. 2024Tighter Generalisation Bounds via Interpolationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 6.

13 papers.

Sort:

  1. 2025Information-Theoretic Framework for Understanding Modern Machine-Learningcited by 2
  2. 2025Model Merging via Multi-Teacher Knowledge Distillationcited by 2
  3. 2024Isoperimetry is All We Need: Langevin Posterior Sampling for RL with Sublinear Regretcited by 1
  4. 2025Aggregation on Learnable Manifolds for Asynchronous Federated Optimizationcited by 1
  5. 2025A Flat Minima Perspective on Understanding Augmentations and Model Robustnesscited by 1
  6. 2025Non-Vacuous Generalization Bounds: Can Rescaling Invariances Help?cited by 1
  7. 2026Smoothness-Based Derandomization of PAC-Bayes Boundscited by 1
  8. 2025The Interplay of Statistics and Noisy Optimization: Learning Linear Predictors with Random Data Weightscited by 0
  9. 2026Generalization at the Edge of Stabilitycited by 0
  10. 2026A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learningcited by 0
  11. 2026Certification of Machine Learning Models via Directional Sharpnesscited by 0
  12. 2026Dimensionless Controls of Plasticity Under Alternating Tasks: From Evolutionary Biology to Continual Learningcited by 0
1 more
  1. 2026Distribution of hitting times for dissipative random dynamical systems on ℝd, with application to stochastic gradient descentcited by 0