mlsanityData to 7 Oct 2026

Understanding the double descent curve in Machine Learning

arXiv 2211.10322 ↗First posted 18 Nov 2022cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

The theory of bias-variance used to serve as a guide for model selection when applying Machine Learning algorithms. However, modern practice has shown success with over-parameterized models that were expected to overfit but did not. This led to the proposal of the double descent curve of performance by Belkin et al. Although it seems to describe a real, representative phenomenon, the field is lacking a fundamental theoretical understanding of what is happening, what are the consequences for model selection and when is double descent expected to occur. In this paper we develop a principled understanding of the phenomenon, and sketch answers to these important questions. Furthermore, we report real experimental results that are correctly predicted by our proposed hypothesis.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 14 of its 23 references that are in the map. First posted on arXiv 18 Nov 2022.

Builds on

14 of 23 references.

References · 14

  1. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  2. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  3. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  4. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
  5. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 410
  6. 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
  7. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
  8. 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 260
  9. 2017High-dimensional dynamics of generalization error in neural networkscited by 238
  10. 2019Towards moderate overparameterization: global convergence guarantees for training shallow neural networkscited by 176
  11. 2020Rethinking Bias-Variance Trade-off for Generalization of Neural Networkscited by 107
  12. 2018A jamming transition from under- to over-parametrization affects loss landscape and generalizationcited by 65
2 more
  1. 2018A Modern Take on the Bias-Variance Tradeoff in Neural Networkscited by 64
  2. 2019On the Bias-Variance Tradeoff: Textbooks Need an Updatecited by 8

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 1.

2 papers.

  1. 2022The smooth output assumption, and why deep networks are better than wide onescited by 0
  2. 2025Evaluating Double Descent in Machine Learning: Insights from Tree-Based Models Applied to a Genomic Prediction Taskcited by 0