mlsanityData to 7 Oct 2026

High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Models

arXiv 2204.04476 ↗First posted 9 Apr 2022math.ST cs.LG math.PRcited by 2

Not on the map. Why?

Abstract (arXiv)

We study Langevin dynamics for recovering the planted signal in the spiked matrix model. We provide a "path-wise" characterization of the overlap between the output of the Langevin algorithm and the planted signal. This overlap is characterized in terms of a self-consistent system of integro-differential equations, usually referred to as the Crisanti-Horner-Sommers-Cugliandolo-Kurchan (CHSCK) equations in the spin glass literature. As a second contribution, we derive an explicit formula for the limiting overlap in terms of the signal-to-noise ratio and the injected noise in the diffusion. This uncovers a sharp phase transition -- in one regime, the limiting overlap is strictly positive, while in the other, the injected noise overcomes the signal, and the limiting overlap is zero.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 31 of its 59 references that are in the map. First posted on arXiv 9 Apr 2022.

Builds on

31 of 59 references.

References · 31

  1. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  2. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  3. 2019Implicit Regularization in Deep Matrix Factorizationcited by 402
  4. 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
  5. 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 308
  6. 2014The Loss Surfaces of Multilayer Networkscited by 278
  7. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 267
  8. 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 260
  9. 2018Characterizing Implicit Bias in Terms of Optimization Geometrycited by 259
  10. 2018Implicit Bias of Gradient Descent on Linear Convolutional Networkscited by 251
  11. 2018Gradient descent aligns the layers of deep linear networkscited by 182
  12. 2019Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networkscited by 162
19 more
  1. 2017The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learningcited by 150
  2. 2018Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernelcited by 131
  3. 2020Online stochastic gradient descent on non-convex losses from high-dimensional inferencecited by 119
  4. 2018Convergence of Gradient Descent on Separable Datacited by 108
  5. 2018Does data interpolation contradict statistical optimality?cited by 95
  6. 2019A Model of Double Descent for High-dimensional Binary Linear Classificationcited by 81
  7. 2020Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regimecited by 81
  8. 2018A Continuous-Time View of Early Stopping for Least Squarescited by 67
  9. 2020Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classificationcited by 60
  10. 2020The Implicit Regularization of Stochastic Gradient Flow for Least Squarescited by 58
  11. 2020A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-ℓ1-Norm Interpolated Classifierscited by 44
  12. 2019Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Modelscited by 28
  13. 2019Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Modelcited by 25
  14. 2018Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inferencecited by 24
  15. 2021Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature modelcited by 17
  16. 2020When does gradient descent with logistic loss find interpolating two-layer networks?cited by 12
  17. 2021Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problemscited by 5
  18. 2021Rank-one matrix estimation: analytic time evolution of gradient descent dynamicscited by 5
  19. 2020Thresholds of descending algorithms in inference problemscited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1.

2 papers.

  1. 2022High-dimensional limit theorems for SGD: Effective dynamics and critical scalingcited by 69
  2. 2023Gradient flow on extensive-rank positive semi-definite matrix denoisingcited by 1