# The Global Optimization Geometry of Shallow Linear Neural Networks

arXiv: https://arxiv.org/abs/1805.04938 · HTML: https://mlsanity.com/paper/1805.04938/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 May 2018.

## Built on by (most cited first, 11 of 11)

- [A Geometric Analysis of Neural Collapse with Unconstrained Features](https://mlsanity.com/paper/2105.02375/) (2021)
- [The loss landscape of deep linear neural networks: a second-order analysis](https://mlsanity.com/paper/2107.13289/) (2021)
- [Function Space and Critical Points of Linear Convolutional Networks](https://mlsanity.com/paper/2304.05752/) (2023)
- [On Feature Learning in Neural Networks with Global Convergence Guarantees](https://mlsanity.com/paper/2204.10782/) (2022)
- [The Persistence of Neural Collapse Despite Low-Rank Bias](https://mlsanity.com/paper/2410.23169/) (2024)
- [Dynamical stability and chaos in artificial neural network trajectories along training](https://mlsanity.com/paper/2404.05782/) (2024)
- [Beyond Procrustes: Balancing-Free Gradient Descent for Asymmetric Low-Rank Matrix Sensing](https://mlsanity.com/paper/2101.05113/) (2021)
- [OMASGAN: Out-of-Distribution Minimum Anomaly Score GAN for Sample Generation on the Boundary](https://mlsanity.com/paper/2110.15273/) (2021)
- [Memorization-Dilation: Modeling Neural Collapse Under Label Noise](https://mlsanity.com/paper/2206.05530/) (2022)
- [Convergence Analysis for Learning Orthonormal Deep Linear Neural Networks](https://mlsanity.com/paper/2311.14658/) (2023)
- [Two-Layer Linear Auto-Regressive Models Estimate Latent States](https://mlsanity.com/paper/2606.12691/) (2026)

## Abstract (arXiv)

We examine the squared error loss landscape of shallow linear neural networks. We show---with significantly milder assumptions than previous works---that the corresponding optimization problems have benign geometric properties: there are no spurious local minima and the Hessian at every saddle point has at least one negative eigenvalue. This means that at every saddle point there is a directional negative curvature which algorithms can utilize to further decrease the objective value. These geometric properties imply that many local search algorithms (such as the gradient descent which is widely utilized for training neural networks) can provably solve the training problem with global convergence.