mlsanityData to 7 Oct 2026

A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks

arXiv 1707.09564 ↗First posted 29 Jul 2017cs.LGcited by 309

Not on the map. Why?

Abstract (arXiv)

We present a generalization bound for feedforward neural networks in terms of the product of the spectral norm of the layers and the Frobenius norm of the weights. The generalization bound is derived using a PAC-Bayes analysis.

Key facts. Cited by 309 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 66; 2022: 64; 2023: 53; 2024: 43; 2025: 40; 2026: 42.

309 papers.

Sort:

By area:

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  2. 2022Surrogate Gap Minimization Improves Sharpness-Aware Trainingcited by 171
  3. 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 156
  4. 2021Inductive Biases and Variable Creation in Self-Attention Mechanismscited by 144
  5. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  6. 2021Assessing Generalization of SGD via Disagreementcited by 116
  7. 2023What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalizationcited by 112
  8. 2021Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformerscited by 101
  9. 2022Sharpness-Aware Training for Freecited by 95
  10. 2022Benign Overfitting in Two-layer Convolutional Neural Networkscited by 90
  11. 2021Information-Theoretic Generalization Bounds for Stochastic Gradient Descentcited by 84
  12. 2021Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Biascited by 82
38 more
  1. 2022PAC-Bayes Compression Bounds So Tight That They Can Explain Generalizationcited by 75
  2. 2021Subgroup Generalization and Fairness of Graph Neural Networkscited by 69
  3. 2022Contrastive Adapters for Foundation Model Group Robustnesscited by 62
  4. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
  5. 2021Initialization and Regularization of Factorized Neural Layerscited by 58
  6. 2021When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?cited by 57
  7. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  8. 2021Improved Regularization and Robustness for Fine-tuning in Neural Networkscited by 45
  9. 2023Approximation-Generalization Trade-offs under (Approximate) Group Equivariancecited by 45
  10. 2021Heavy Tails in SGD and Compressibility of Overparametrized Neural Networkscited by 44
  11. 2022Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 42
  12. 2023Implicit Bias of Gradient Descent for Logistic Regression at the Edge of Stabilitycited by 42
  13. 2022Anticorrelated Noise Injection for Improved Generalizationcited by 41
  14. 2023Demystifying Structural Disparity in Graph Neural Networks: Can One Size Fit All?cited by 41
  15. 2023Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusioncited by 39
  16. 2021Understanding the role of importance weighting for deep learningcited by 38
  17. 2021Rethinking Graph Auto-Encoder Models for Attributed Graph Clusteringcited by 37
  18. 2021Taxonomizing local versus global structure in neural network loss landscapescited by 34
  19. 2022Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guaranteescited by 34
  20. 2022An Impartial Take to the CNN vs Transformer Robustness Contestcited by 34
  21. 2023Benign Overfitting for Two-layer ReLU Convolutional Neural Networkscited by 34
  22. 2024Why Do You Grok? A Theoretical Analysis of Grokking Modular Additioncited by 34
  23. 2023Learning to Warm-Start Fixed-Point Optimization Algorithmscited by 32
  24. 2021On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implicationscited by 31
  25. 2023On the Stability of Expressive Positional Encodings for Graphscited by 31
  26. 2021Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stabilitycited by 29
  27. 2021Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalizationcited by 29
  28. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  29. 2021Scaling Laws for Deep Learningcited by 25
  30. 2022On the Generalization Mystery in Deep Learningcited by 25
  31. 2022Explicit Regularization in Overparametrized Models via Noise Injectioncited by 25
  32. 2021Measuring Generalization with Optimal Transportcited by 24
  33. 2021Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detectioncited by 23
  34. 2021Formalizing Generalization and Robustness of Neural Networks to Weight Perturbationscited by 22
  35. 2022Non-Vacuous Generalisation Bounds for Shallow Neural Networkscited by 22
  36. 2022Rate-Distortion Theoretic Generalization Bounds for Stochastic Learning Algorithmscited by 22
  37. 2022Approximation bounds for norm constrained neural networks with applications to regression and GANscited by 21
  38. 2023Generalization bounds for neural ordinary differential equations and deep residual networkscited by 21

50 most cited of 309 All 309 (JSON)