mlsanityData to 7 Oct 2026

Gradient flow on extensive-rank positive semi-definite matrix denoising

arXiv 2303.09474 ↗First posted 16 Mar 2023stat.ML cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

In this work, we present a new approach to analyze the gradient flow for a positive semi-definite matrix denoising problem in an extensive-rank and high-dimensional regime. We use recent linear pencil techniques of random matrix theory to derive fixed point equations which track the complete time evolution of the matrix-mean-square-error of the problem. The predictions of the resulting fixed point equations are validated by numerical experiments. In this short note we briefly illustrate a few predictions of our formalism by way of examples, and in particular we uncover continuous phase transitions in the extensive-rank and high-dimensional regime, which connect to the classical phase transitions of the low-rank problem in the appropriate limit. The formalism has much wider applicability than shown in this communication.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 37 references that are in the map. First posted on arXiv 16 Mar 2023.

Builds on

18 of 37 references.

References · 18

  1. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  2. 2017Implicit Regularization in Matrix Factorizationcited by 318
  3. 2018Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overviewcited by 159
  4. 2020The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalizationcited by 101
  5. 2014A statistical model for tensor PCAcited by 72
  6. 2022High-dimensional limit theorems for SGD: Effective dynamics and critical scalingcited by 69
  7. 2020Gradient Descent for Deep Matrix Factorization: Dynamics and Implicit Bias towards Low Rankcited by 42
  8. 2017Estimation of Low-Rank Matrices via Approximate Message Passingcited by 41
  9. 2019Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Modelscited by 28
  10. 2018Harnessing Structures in Big Data via Guaranteed Low-Rank Matrix Estimationcited by 25
  11. 2014Phase transitions and sample complexity in Bayes-optimal matrix factorizationcited by 22
  12. 2021Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature modelcited by 17
6 more
  1. 2021Statistical limits of dictionary learning: random matrix theory and the spectral replica methodcited by 10
  2. 2022The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?cited by 9
  3. 2022Matrix factorization with neural networkscited by 6
  4. 2021Rank-one matrix estimation: analytic time evolution of gradient descent dynamicscited by 5
  5. 2022Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structurescited by 5
  6. 2022High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Modelscited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.