Gradient descent in matrix factorization: Understanding large initialization
Not on the map. Why?
Abstract (arXiv)
Gradient Descent (GD) has been proven effective in solving various matrix factorization problems. However, its optimization behavior with large initial values remains less understood. To address this gap, this paper presents a novel theoretical framework for examining the convergence trajectory of GD with a large initialization. The framework is grounded in signal-to-noise ratio concepts and inductive arguments. The results uncover an implicit incremental learning phenomenon in GD and offer a deeper understanding of its performance in large initialization scenarios.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 11 of its 22 references that are in the map. First posted on arXiv 30 May 2023.
Builds on
11 of 22 references.
References · 11
- 2018Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overviewcited by 160
- 2020Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learningcited by 150
- 2016A Geometric Analysis of Phase Retrievalcited by 92
- 2021Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstructioncited by 78
- 2018Gradient Descent with Random Initialization: Fast Global Convergence for Nonconvex Phase Retrievalcited by 71
- 2015Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picturecited by 49
- 2023Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensingcited by 42
- 2017Global Optimality in Low-rank Matrix Optimizationcited by 36
- 2022Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorizationcited by 34
- 2017The Global Optimization Geometry of Low-Rank Matrix Optimizationcited by 25
- 2022Behind the Scenes of Gradient Descent: A Trajectory Analysis via Basis Function Decompositioncited by 4
Built on by
Citing papers per year: 2025: 2.
2 papers.