mlsanityData to 14 Oct 2026

Size-Independent Sample Complexity of Neural Networks

arXiv 1712.06541 ↗First posted 18 Dec 2017cs.LG cs.NE stat.MLcited by 290

Not on the map. Why?

Abstract (arXiv)

We study the sample complexity of learning neural networks, by providing new bounds on their Rademacher complexity assuming norm constraints on the parameter matrix of each layer. Compared to previous work, these complexity bounds have improved dependence on the network depth, and under some additional assumptions, are fully independent of the network size (both depth and width). These results are derived using some novel techniques, which may be of independent interest.

Key facts. Cited by 290 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 55; 2022: 52; 2023: 65; 2024: 37; 2025: 43; 2026: 38.

290 papers.

Sort:

By area:

  1. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 313
  2. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
  3. 2021Inductive Biases and Variable Creation in Self-Attention Mechanismscited by 144
  4. 2021Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformerscited by 102
  5. 2021On the Role of Neural Collapse in Transfer Learningcited by 100
  6. 2022Neural Collapse: A Review on Modelling Principles and Generalizationcited by 95
  7. 2022Benign Overfitting in Two-layer Convolutional Neural Networkscited by 91
  8. 2022On the Implicit Bias in Deep-Learning Algorithmscited by 86
  9. 2021ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reductioncited by 84
  10. 2021When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?cited by 58
  11. 2021Causally motivated Shortcut Removal Using Auxiliary Labelscited by 55
  12. 2023Towards Label-free Scene Understanding by Vision Foundation Modelscited by 54
38 more
  1. 2023Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusioncited by 40
  2. 2021Understanding the role of importance weighting for deep learningcited by 39
  3. 2023Learning to Augment Distributions for Out-of-Distribution Detectioncited by 38
  4. 2021Learning with Multiclass AUC: Theory and Algorithmscited by 35
  5. 2021Generalization bounds via distillationcited by 34
  6. 2023Reverse Engineering Self-Supervised Learningcited by 33
  7. 2022Combining Observational and Randomized Data for Estimating Heterogeneous Treatment Effectscited by 32
  8. 2023Sequence Length Independent Norm-Based Generalization Bounds for Transformerscited by 29
  9. 2021Defending Against Image Corruptions Through Adversarial Augmentationscited by 28
  10. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 27
  11. 2021Stability & Generalisation of Gradient Descent for Shallow Neural Networks without the Neural Tangent Kernelcited by 26
  12. 2021On the statistical complexity of quantum circuitscited by 25
  13. 2021Measuring Generalization with Optimal Transportcited by 25
  14. 2022On the Generalization Mystery in Deep Learningcited by 25
  15. 2023Generalization Analysis for Contrastive Representation Learningcited by 25
  16. 2023An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularizationcited by 25
  17. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  18. 2021Formalizing Generalization and Robustness of Neural Networks to Weight Perturbationscited by 22
  19. 2021Understanding the Generalization Benefit of Model Invariance from a Data Perspectivecited by 22
  20. 2022Approximation bounds for norm constrained neural networks with applications to regression and GANscited by 22
  21. 2021Representation Learning Beyond Linear Prediction Functionscited by 21
  22. 2022Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networkscited by 21
  23. 2023Generalization and Stability of Interpolating Neural Networks with Minimal Widthcited by 21
  24. 2023Generalization bounds for neural ordinary differential equations and deep residual networkscited by 21
  25. 2023Large Language Model-Enhanced Algorithm Selection: Towards Comprehensive Algorithm Representationcited by 21
  26. 2021Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrectioncited by 20
  27. 2025Pareto Set Learning for Multi-Objective Reinforcement Learningcited by 20
  28. 2025Sparse identification of nonlinear dynamics and Koopman operators with Shallow Recurrent Decoder Networkscited by 20
  29. 2021Learning from Similarity-Confidence Datacited by 19
  30. 2023Fantastic Generalization Measures are Nowhere to be Foundcited by 19
  31. 2022A PAC-Bayesian Generalization Bound for Equivariant Networkscited by 18
  32. 2022Adversarial Rademacher Complexity of Deep Neural Networkscited by 18
  33. 2024How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teacherscited by 18
  34. 2021Generalization bounds for graph convolutional neural networks via Rademacher complexitycited by 17
  35. 2024Can AI Be as Creative as Humans?cited by 17
  36. 2022On Rademacher Complexity-based Generalization Bounds for Deep Learningcited by 16
  37. 2022Theoretical analysis of deep neural networks for temporally dependent observationscited by 16
  38. 2021Beyond Self-Supervision: A Simple Yet Effective Network Distillation Alternative to Improve Backbonescited by 15

50 most cited of 290 All 290 (JSON)