mlsanityData to 14 Oct 2026

Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models

arXiv 2502.05074 ↗First posted 7 Feb 2025cond-mat.dis-nn cs.LG stat.MLcited by 7

Not on the map. Why?

Abstract (arXiv)

We derive a novel deterministic equivalence for the two-point function of a random matrix resolvent. Using this result, we give a unified derivation of the performance of a wide variety of high-dimensional linear models trained with stochastic gradient descent. This includes high-dimensional linear regression, kernel regression, and linear random feature models. Our results include previously known asymptotics as well as novel ones.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 31 of its 44 references that are in the map. First posted on arXiv 7 Feb 2025.

Builds on

31 of 44 references.

References · 31

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  3. 2021Scaling Vision Transformerscited by 999
  4. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 513
  5. 2023Scaling Data-Constrained Language Modelscited by 403
  6. 2021Scaling Laws for Transfercited by 272
  7. 2020Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networkscited by 166
  8. 2022Scaling Laws and Interpretability of Learning from Repeated Datacited by 152
  9. 2021Learning curves of generic features maps for realistic datasets with a teacher-student modelcited by 126
  10. 2021Scaling Laws for Neural Machine Translationcited by 111
  11. 2024A Dynamical Model of Neural Scaling Lawscited by 111
  12. 2020The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalizationcited by 101
19 more
  1. 2023Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Designcited by 89
  2. 2017A Random Matrix Approach to Neural Networkscited by 80
  3. 2024Scaling Exponents Across Parameterizations and Optimizerscited by 76
  4. 2018Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passescited by 69
  5. 20244+3 Phases of Compute-Optimal Neural Scaling Lawscited by 68
  6. 2024Scaling Laws in Linear Regression: Compute, Parameters, and Datacited by 57
  7. 2024Scaling and renormalization in high-dimensional regressioncited by 50
  8. 2023High-dimensional analysis of double descent for linear regression with random projectionscited by 47
  9. 2023Scaling MLPs: A Tale of Inductive Biascited by 37
  10. 2021SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticalitycited by 35
  11. 2023Deterministic equivalent and error universality of deep random features learningcited by 35
  12. 2018Learning with SGD and Random Featurescited by 33
  13. 2024A non-asymptotic theory of Kernel Ridge Regression: deterministic equivalents, test error, and GCV estimatorcited by 28
  14. 2022Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensionscited by 23
  15. 2021Out-of-Distribution Generalization in Kernel Regressioncited by 21
  16. 2024Optimal Ridge Regularization for Out-of-Distribution Predictioncited by 19
  17. 2023Learning curves for deep structured Gaussian feature modelscited by 13
  18. 2022A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momentacited by 11
  19. 2024Risk and cross validation in ridge regression with correlated samplescited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Timecited by 16
  2. 2025Models of Heavy-Tailed Mechanistic Universalitycited by 10
  3. 2026Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averagingcited by 10
  4. 2025A Simplified Analysis of SGD for Linear Regression with Weight Averagingcited by 4
  5. 2026An Asymptotic Theory of Chain-of-Thought in In-Context Learningcited by 0
  6. 2026Beyond Negative-Ridge Endpoints: Mixed-Sign Spectral Regularization via Negative-Shifted Gradient Descentcited by 0
  7. 2026An Active-Bottleneck Mechanism for Weak-to-Strong Generalizationcited by 0