mlsanityData to 7 Oct 2026

Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models

arXiv 1902.00139 ↗First posted 1 Feb 2019cs.LG cond-mat.dis-nn math.STcited by 28

Not on the map. Why?

Abstract (arXiv)

In this work we analyse quantitatively the interplay between the loss landscape and performance of descent algorithms in a prototypical inference problem, the spiked matrix-tensor model. We study a loss function that is the negative log-likelihood of the model. We analyse the number of local minima at a fixed distance from the signal/spike with the Kac-Rice formula, and locate trivialization of the landscape at large signal-to-noise ratios. We evaluate in a closed form the performance of a gradient flow algorithm using integro-differential PDEs as developed in physics of disordered systems for the Langevin dynamics. We analyze the performance of an approximate message passing algorithm estimating the maximum likelihood configuration via its state evolution. We conclude by comparing the above results: while we observe a drastic slow down of the gradient flow dynamics even in the region where the landscape is trivial, both the analyzed algorithms are shown to perform well even in the part of the region of parameters where spurious local minima are present.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 6; 2023: 3; 2024: 7; 2025: 3; 2026: 3.

28 papers.

Sort:

  1. 2024A Dynamical Model of Neural Scaling Lawscited by 105
  2. 2022Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networkscited by 104
  3. 2022Learning Single-Index Models with Shallow Neural Networkscited by 92
  4. 2024How Feature Learning Can Improve Neural Scaling Lawscited by 55
  5. 2024The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponentscited by 50
  6. 2024Infinite Limits of Multi-head Transformer Dynamicscited by 38
  7. 2021An Analytical Theory of Curriculum Learning in Teacher-Student Networkscited by 27
  8. 2025Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfercited by 17
  9. 2024Sliding down the stairs: how correlated latent variables accelerate learning with neural networkscited by 12
  10. 2021Statistical limits of dictionary learning: random matrix theory and the spectral replica methodcited by 10
  11. 2022Sudakov-Fernique post-AMP, and a new proof of the local convexity of the TAP free energycited by 9
  12. 2024The Role of the Time-Dependent Hessian in High-Dimensional Optimizationcited by 7
16 more
  1. 2022On free energy barriers in Gaussian priors and failure of cold start MCMC for high-dimensional unimodal distributionscited by 6
  2. 2022Average-Case Complexity of Tensor Decomposition for Low-Degree Polynomialscited by 6
  3. 2021Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problemscited by 5
  4. 2021Rank-one matrix estimation: analytic time evolution of gradient descent dynamicscited by 5
  5. 2025Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Modelcited by 5
  6. 2021A spin-glass model for the loss surfaces of generative adversarial networkscited by 4
  7. 2023Stochastic Gradient Descent outperforms Gradient Descent in recovering a high-dimensional signal in a glassy energy landscapecited by 3
  8. 2024Asymptotic Dynamics of Alternating Minimization for Bilinear Regressioncited by 3
  9. 2022High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Modelscited by 2
  10. 2025Overparametrization bends the landscape: BBP transitions at initialization in simple Neural Networkscited by 2
  11. 2026Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networkscited by 2
  12. 2023Gradient flow on extensive-rank positive semi-definite matrix denoisingcited by 1
  13. 2021Selective Multiple Power Iteration: from Tensor PCA to gradient-based exploration of landscapescited by 0
  14. 2023Topological complexity of spiked random polynomials and finite-rank spherical integralscited by 0
  15. 2026Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrievalcited by 0
  16. 2026Theory of learning of high-dimensional controlled non-linear dynamical systems (I): models and methodscited by 0