mlsanityData to 14 Oct 2026

A Margin-based Multiclass Generalization Bound via Geometric Complexity

arXiv 2405.18590 ↗First posted 28 May 2024stat.ML cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

There has been considerable effort to better understand the generalization capabilities of deep neural networks both as a means to unlock a theoretical understanding of their success as well as providing directions for further improvements. In this paper, we investigate margin-based multiclass generalization bounds for neural networks which rely on a recent complexity measure, the geometric complexity, developed for neural networks. We derive a new upper bound on the generalization error which scales with the margin-normalized geometric complexity of the network and which holds for a broad family of data distributions and model classes. Our generalization bound is empirically investigated for a ResNet-18 model trained with SGD on the CIFAR-10 and CIFAR-100 datasets with both original and random labels.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 24 of its 54 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

24 of 54 references.

References · 24

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  3. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  4. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  5. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  6. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  7. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  8. 2019Fantastic Generalization Measures and Where to Find Themcited by 513
  9. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 314
  10. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 224
  11. 2017Emergence of Invariance and Disentanglement in Deep Representationscited by 215
  12. 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 189
12 more
  1. 2021On the Origin of Implicit Regularization in Stochastic Gradient Descentcited by 184
  2. 2021A Universal Law of Robustness via Isoperimetrycited by 151
  3. 2020Implicit Gradient Regularizationcited by 146
  4. 2021Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolationcited by 132
  5. 2016Robust Large Margin Deep Neural Networkscited by 118
  6. 2017Implicit Regularization in Deep Learningcited by 90
  7. 2019Generalization bounds for deep convolutional neural networkscited by 59
  8. 2019The intriguing role of module criticality in the generalization of deep networkscited by 40
  9. 2022Why neural networks find simple solutions: the many regularizers of geometric complexitycited by 39
  10. 2023Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descentcited by 25
  11. 2022Deep Double Descent via Smooth Interpolationcited by 8
  12. 2021The Geometric Occam's Razor Implicit in Deep Learningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Penalizing Localized Dirichlet Energies in Low Rank Tensor Productscited by 0
  2. 2026The Price of Robustness: Stable Classifiers Need Overparameterizationcited by 0