mlsanityData to 7 Oct 2026

Multiple Descent: Design Your Own Generalization Curve

arXiv 2008.01036 ↗First posted 3 Aug 2020cs.LG math.ST stat.MLcited by 44

Not on the map. Why?

Abstract (arXiv)

This paper explores the generalization loss of linear regression in variably parameterized families of models, both under-parameterized and over-parameterized. We show that the generalization curve can have an arbitrary number of peaks, and moreover, locations of those peaks can be explicitly controlled. Our results highlight the fact that both classical U-shaped generalization curve and the recently observed double descent curve are not intrinsic properties of the model family. Instead, their emergence is due to the interaction between the properties of the data and the inductive biases of learning algorithms.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 16; 2023: 9; 2024: 6; 2025: 1; 2026: 2.

44 papers.

Sort:

  1. 2022Omnigrok: Grokking Beyond Algorithmic Datacited by 142
  2. 2021The Shape of Learning Curves: a Reviewcited by 67
  3. 2021Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 42
  4. 2022Second-order regression models exhibit progressive sharpening to the edge of stabilitycited by 40
  5. 2023Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzlecited by 36
  6. 2023Deep networks for system identification: a Surveycited by 35
  7. 2021Taxonomizing local versus global structure in neural network loss landscapescited by 34
  8. 2023Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grokcited by 31
  9. 2021Towards an Understanding of Benign Overfitting in Neural Networkscited by 29
  10. 2021Multi-scale Feature Learning Dynamics: Insights for Double Descentcited by 24
  11. 2021Out-of-Distribution Generalization in Kernel Regressioncited by 21
  12. 2022Fluctuations, Bias, Variance & Ensemble of Learners: Exact Asymptotics for Convex Losses in High-Dimensioncited by 19
32 more
  1. 2022Homophily modulates double descent generalization in graph convolution networkscited by 13
  2. 2021On the interplay between data structure and loss function in classification problemscited by 12
  3. 2022A Non-Asymptotic Moreau Envelope Theory for High-Dimensional Generalized Linear Modelscited by 11
  4. 2022Multiple Descent in the Multiple Random Feature Modelcited by 10
  5. 2022Mitigating multiple descents: A model-agnostic framework for risk monotonizationcited by 9
  6. 2022Information bottleneck theory of high-dimensional regression: relevancy, efficiency and optimalitycited by 9
  7. 2024Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyondcited by 9
  8. 2021On the Double Descent of Random Features Models Trained with SGDcited by 5
  9. 2021Minimum ℓ1-norm interpolators: Precise asymptotics and multiple descentcited by 5
  10. 2022Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structurescited by 5
  11. 2022On Optimal Early Stopping: Over-informative versus Under-informative Parametrizationcited by 4
  12. 2023Extrapolated cross-validation for randomized ensemblescited by 4
  13. 2023Finding Regularized Competitive Equilibria of Heterogeneous Agent Macroeconomic Models with Reinforcement Learningcited by 4
  14. 2023Least Squares Regression Can Exhibit Under-Parameterized Double Descentcited by 4
  15. 2023A PAC-Bayesian Perspective on the Interpolating Information Criterioncited by 4
  16. 2021Mitigating deep double descent by concatenating inputscited by 3
  17. 2024Understanding the Double Descent Phenomenon in Deep Learningcited by 3
  18. 2024Unveiling Multiple Descents in Unsupervised Autoencoderscited by 3
  19. 2022Cascaded Gaps: Towards Gap-Dependent Regret for Risk-Sensitive Reinforcement Learningcited by 2
  20. 2023Dropout Drops Double Descentcited by 2
  21. 2024Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Settingcited by 2
  22. 2025A dynamic view of some anomalous phenomena in SGDcited by 2
  23. 2022Benefit of Interpolation in Nearest Neighbor Algorithmscited by 1
  24. 2022The BUTTER Zone: An Empirical Study of Training Dynamics in Fully Connected Neural Networkscited by 1
  25. 2023Transgressing the boundaries: towards a rigorous understanding of deep learning and its (non-)robustnesscited by 1
  26. 2022With Greater Distance Comes Worse Performance: On the Perspective of Layer Utilization and Model Generalizationcited by 0
  27. 2022Improving Students' Academic Performance with AI and Semantic Technologiescited by 0
  28. 2022Regularization Trade-offs with Fake Featurescited by 0
  29. 2024eGAD! double descent is explained by Generalized Aliasing Decompositioncited by 0
  30. 2024Investigating the Impact of Model Complexity in Large Language Modelscited by 0
  31. 2026Deep Exploration of Epoch-wise Double Descent in Noisy Data: Signal Separation, Large Activation, and Benign Overfittingcited by 0
  32. 2026A Regularization-Sharpness Tradeoff for Linear Interpolatorscited by 0