mlsanityData to 14 Oct 2026

The Double Descent Behavior in Two Layer Neural Network for Binary Classification

arXiv 2504.19351 ↗First posted 27 Apr 2025stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent studies observed a surprising concept on model test error called the double descent phenomenon, where the increasing model complexity decreases the test error first and then the error increases and decreases again. To observe this, we work on a two layer neural network model with a ReLU activation function designed for binary classification under supervised learning. Our aim is to observe and investigate the mathematical theory behind the double descent behavior of model test error for varying model sizes. We quantify the model size by the ratio of number of training samples to the dimension of the model. Due to the complexity of the empirical risk minimization procedure, we use the Convex Gaussian Min Max Theorem to find a suitable candidate for the global training loss.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 19 references that are in the map. First posted on arXiv 27 Apr 2025.

Builds on

12 of 19 references.

References · 12

  1. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  2. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  3. 2017High-dimensional dynamics of generalization error in neural networkscited by 241
  4. 2020Optimal Regularization Can Mitigate Double Descentcited by 109
  5. 2020Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regimecited by 95
  6. 2019Scaling description of generalization with number of parameters in deep learningcited by 83
  7. 2019A Model of Double Descent for High-dimensional Binary Linear Classificationcited by 81
  8. 2018A jamming transition from under- to over-parametrization affects loss landscape and generalizationcited by 67
  9. 2020The role of regularization in classification of high-dimensional noisy Gaussian mixturecited by 63
  10. 2021SGD Generalizes Better Than GD (And Regularization Doesn't Help)cited by 38
  11. 2019More Data Can Hurt for Linear Regression: Sample-wise Double Descentcited by 37
  12. 2020Analytic Study of Double Descent in Binary Classification: The Impact of Losscited by 28

Built on by

Not cited yet.