mlsanityData to 14 Oct 2026

Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Model

arXiv 1907.08226 ↗First posted 18 Jul 2019cs.LG cond-mat.dis-nn math.STcited by 25

Not on the map. Why?

Abstract (arXiv)

Gradient-based algorithms are effective for many machine learning tasks, but despite ample recent effort and some progress, it often remains unclear why they work in practice in optimising high-dimensional non-convex functions and why they find good minima instead of being trapped in spurious ones. Here we present a quantitative theory explaining this behaviour in a spiked matrix-tensor model. Our framework is based on the Kac-Rice analysis of stationary points and a closed-form analysis of gradient-flow originating from statistical physics. We show that there is a well defined region of parameters where the gradient-flow algorithm finds a good global minimum despite the presence of exponentially many spurious local minima. We show that this is achieved by surfing on saddles that have strong negative direction towards the global minima, a phenomenon that is connected to a BBP-type threshold in the Hessian describing the critical points of the landscapes.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 6; 2023: 2; 2024: 3; 2025: 3; 2026: 2.

25 papers.

Sort:

By area:

  1. 2022High-dimensional limit theorems for SGD: Effective dynamics and critical scalingcited by 70
  2. 2024The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponentscited by 52
  3. 2021An Analytical Theory of Curriculum Learning in Teacher-Student Networkscited by 27
  4. 2023On the Impact of Overparameterization on the Training of a Shallow Neural Network in High Dimensionscited by 21
  5. 2024Sliding down the stairs: how correlated latent variables accelerate learning with neural networkscited by 12
  6. 2021Statistical limits of dictionary learning: random matrix theory and the spectral replica methodcited by 10
  7. 2021Long Random Matrices and Tensor Unfoldingcited by 10
  8. 2022Sudakov-Fernique post-AMP, and a new proof of the local convexity of the TAP free energycited by 9
  9. 2021Tilting the playing field: Dynamical loss functions for machine learningcited by 7
  10. 2021Appearance of Random Matrix Theory in Deep Learningcited by 7
  11. 2022On free energy barriers in Gaussian priors and failure of cold start MCMC for high-dimensional unimodal distributionscited by 6
  12. 2025Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Modelcited by 6
13 more
  1. 2021Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problemscited by 5
  2. 2021Rank-one matrix estimation: analytic time evolution of gradient descent dynamicscited by 5
  3. 2021A spin-glass model for the loss surfaces of generative adversarial networkscited by 4
  4. 2022Optimal learning rate schedules in high-dimensional non-convex optimization problemscited by 4
  5. 2022Universal characteristics of deep neural network loss surfaces from random matrix theorycited by 4
  6. 2025Flat Channels to Infinity in Neural Loss Landscapescited by 3
  7. 2022High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Modelscited by 2
  8. 2025Overparametrization bends the landscape: BBP transitions at initialization in simple Neural Networkscited by 2
  9. 2026Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networkscited by 2
  10. 2026Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrievalcited by 1
  11. 2021Selective Multiple Power Iteration: from Tensor PCA to gradient-based exploration of landscapescited by 0
  12. 2023Topological complexity of spiked random polynomials and finite-rank spherical integralscited by 0
  13. 2024Dynamical loss functions shape landscape topography and improve learning in artificial neural networkscited by 0