mlsanityData to 7 Oct 2026

Generalization in Deep Learning

arXiv 1710.05468 ↗First posted 16 Oct 2017stat.ML cs.AI cs.LGcited by 146

Not on the map. Why?

Abstract (arXiv)

This paper provides theoretical insights into why and how deep learning can generalize well, despite its large capacity, complexity, possible algorithmic instability, nonrobustness, and sharp minima, responding to an open question in the literature. We also discuss approaches to provide non-vacuous generalization guarantees for deep learning. Based on theoretical observations, we propose new open problems and discuss the limitations of our results.

Key facts. Cited by 146 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 46; 2022: 39; 2023: 22; 2024: 22; 2025: 12; 2026: 5.

146 papers.

Sort:

By area:

  1. 2021Trustworthy AI: From Principles to Practicescited by 206
  2. 2021Deep Learning Through the Lens of Example Difficultycited by 139
  3. 2023How Does Information Bottleneck Help Deep Learning?cited by 92
  4. 2021Model Complexity of Deep Learning: A Surveycited by 64
  5. 2023The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learningcited by 63
  6. 2021When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?cited by 57
  7. 2022Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodologycited by 43
  8. 2021TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereocited by 39
  9. 2021On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layerscited by 35
  10. 2022Differentially Private Decoding in Large Language Modelscited by 34
  11. 2021Embracing the Dark Knowledge: Domain Generalization Using Regularized Knowledge Distillationcited by 28
  12. 2023An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularizationcited by 25
38 more
  1. 2022Implicit regularization of dropoutcited by 24
  2. 2022Robustness Implies Generalization via Data-Dependent Generalization Boundscited by 23
  3. 2022Non-Vacuous Generalisation Bounds for Shallow Neural Networkscited by 22
  4. 2022Random matrix analysis of deep neural network weight matricescited by 21
  5. 2021The dilemma of quantum neural networkscited by 20
  6. 2021Neuron Coverage-Guided Domain Generalizationcited by 19
  7. 2021On Margins and Derandomisation in PAC-Bayescited by 19
  8. 2021Variational Disentanglement for Domain Generalizationcited by 18
  9. 2022Evaluating the Generalization Ability of Super-Resolution Networkscited by 18
  10. 2024Towards Unified Multimodal Editing with Enhanced Knowledge Collaborationcited by 18
  11. 2023Limits for Learning with Language Modelscited by 17
  12. 2024Can AI Be as Creative as Humans?cited by 17
  13. 2022The no-free-lunch theorems of supervised learningcited by 16
  14. 2024A Manifold Perspective on the Statistical Generalization of Graph Neural Networkscited by 16
  15. 2024On Catastrophic Inheritance of Large Foundation Modelscited by 15
  16. 2021The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimationcited by 14
  17. 2023Prediction Error-based Classification for Class-Incremental Learningcited by 14
  18. 2021Robustness to Augmentations as a Generalization metriccited by 13
  19. 2022Are All Linear Regions Created Equal?cited by 13
  20. 2022Two-level Data Augmentation for Calibrated Multi-view Detectioncited by 13
  21. 2021NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networkscited by 12
  22. 2022Effects of Data Geometry in Early Deep Learningcited by 12
  23. 2021Adversarial Feature Augmentation and Normalization for Visual Recognitioncited by 11
  24. 2021Contrastive Learning of Generalized Game Representationscited by 11
  25. 2021Information-Theoretic Characterization of the Generalization Error for Iterative Semi-Supervised Learningcited by 11
  26. 2021PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testingcited by 11
  27. 2021On the Convergence of Shallow Neural Network Training with Randomly Masked Neuronscited by 11
  28. 2022Generalization in Neural Networks: A Broad Surveycited by 11
  29. 2022Learning-based Design of Luenberger Observers for Autonomous Nonlinear Systemscited by 11
  30. 2023Overcoming the Barrier of Orbital-Free Density Functional Theory for Molecular Systems Using Deep Learningcited by 11
  31. 2021What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Datacited by 10
  32. 2023A path-norm toolkit for modern networks: consequences, promises and challengescited by 10
  33. 2024Information-Theoretic Generalization Bounds for Deep Neural Networkscited by 10
  34. 2021Is it enough to optimize CNN architectures on ImageNet?cited by 9
  35. 2022Predicting the generalization gap in neural networks using topological data analysiscited by 9
  36. 2022Deep learning of quantum entanglement from incomplete measurementscited by 9
  37. 2022Boundary between noise and information applied to filtering neural network weight matricescited by 9
  38. 2021A Recipe for Global Convergence Guarantee in Deep Neural Networkscited by 8

50 most cited of 146 All 146 (JSON)