# Asymptotic Escape of Spurious Critical Points on the Low-rank Matrix Manifold

arXiv: https://arxiv.org/abs/2107.09207 · HTML: https://mlsanity.com/paper/2107.09207/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 20 Jul 2021.

## Builds on (most cited first, 12 of 12)

- [Implicit Regularization in Deep Matrix Factorization](https://mlsanity.com/paper/1905.13655/) (2019)
- [Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview](https://mlsanity.com/paper/1809.09573/) (2018)
- [First-order methods almost always avoid saddle points: the case of vanishing step-sizes](https://mlsanity.com/paper/1906.07772/) (2019)
- [Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions](https://mlsanity.com/paper/1605.00405/) (2016)
- [Escaping from saddle points on Riemannian manifolds](https://mlsanity.com/paper/1906.07355/) (2019)
- [Gradient Descent Converges to Minimizers](https://mlsanity.com/paper/1602.04915/) (2016)
- [Efficiently escaping saddle points on manifolds](https://mlsanity.com/paper/1906.04321/) (2019)
- [Convergence results for projected line-search methods on varieties of low-rank matrices via Lojasiewicz inequality](https://mlsanity.com/paper/1402.5284/) (2014)
- [Fast Global Convergence for Low-rank Matrix Recovery via Riemannian Gradient Descent with Random Initialization](https://mlsanity.com/paper/2012.15467/) (2020)
- [Finding stationary points on bounded-rank matrices: A geometric hurdle and a smooth remedy](https://mlsanity.com/paper/2107.03877/) (2021)
- [Analysis of Asymptotic Escape of Strict Saddle Sets in Manifold Optimization](https://mlsanity.com/paper/1911.12518/) (2019)
- [Low-Rank Matrix Optimization Over Affine Set](https://mlsanity.com/paper/1912.03029/) (2019)

## Abstract (arXiv)

We show that on the manifold of fixed-rank and symmetric positive semi-definite matrices, the Riemannian gradient descent algorithm almost surely escapes some spurious critical points on the boundary of the manifold. Our result is the first to partially overcome the incompleteness of the low-rank matrix manifold without changing the vanilla Riemannian gradient descent algorithm. The spurious critical points are some rank-deficient matrices that capture only part of the eigen components of the ground truth. Unlike classical strict saddle points, they exhibit very singular behavior. We show that using the dynamical low-rank approximation and a rescaled gradient flow, some of the spurious critical points can be converted to classical strict saddle points in the parameterized domain, which leads to the desired result. Numerical experiments are provided to support our theoretical findings.