mlsanityData to 7 Oct 2026

VC Theoretical Explanation of Double Descent

arXiv 2205.15549 ↗First posted 31 May 2022stat.ML cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

There has been growing interest in generalization performance of large multilayer neural networks that can be trained to achieve zero training error, while generalizing well on test data. This regime is known as 'second descent' and it appears to contradict the conventional view that optimal model complexity should reflect an optimal balance between underfitting and overfitting, i.e., the bias-variance trade-off. This paper presents a VC-theoretical analysis of double descent and shows that it can be fully explained by classical VC-generalization bounds. We illustrate an application of analytic VC-bounds for modeling double descent for classification, using empirical results for several learning methods, such as SVM, Least Squares, and Multilayer Perceptron classifiers. In addition, we discuss several reasons for the misinterpretation of VC-theoretical results in Deep Learning community.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 17 of its 41 references that are in the map. First posted on arXiv 31 May 2022.

Builds on

17 of 41 references.

References · 17

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  4. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  5. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  6. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  7. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 508
  8. 2017Sharp Minima Can Generalize For Deep Netscited by 462
  9. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
  10. 2017Implicit Regularization in Matrix Factorizationcited by 318
  11. 2021Deep learning: a statistical viewpointcited by 239
  12. 2019Two models of double descent for weak featurescited by 194
5 more
  1. 2018To understand deep learning we need to understand kernel learningcited by 186
  2. 2021Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolationcited by 130
  3. 2016Robust Large Margin Deep Neural Networkscited by 117
  4. 2020Deep Equals Shallow for ReLU Networks in Kernel Regimescited by 83
  5. 2018A Modern Take on the Bias-Variance Tradeoff in Neural Networkscited by 64

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 1.

4 papers.

  1. 2023Understanding the Role of Optimization in Double Descentcited by 0
  2. 2024eGAD! double descent is explained by Generalized Aliasing Decompositioncited by 0
  3. 2024Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexitycited by 0
  4. 2025Evaluating Double Descent in Machine Learning: Insights from Tree-Based Models Applied to a Genomic Prediction Taskcited by 0