mlsanityData to 7 Oct 2026

Landscape Complexity for the Empirical Risk of Generalized Linear Models

arXiv 1912.02143 ↗First posted 4 Dec 2019stat.ML cond-mat.dis-nn cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

We present a method to obtain the average and the typical value of the number of critical points of the empirical risk landscape for generalized linear estimation problems and variants. This represents a substantial extension of previous applications of the Kac-Rice method since it allows to analyze the critical points of high dimensional non-Gaussian random functions. Under a technical hypothesis, we obtain a rigorous explicit variational formula for the annealed complexity, which is the logarithm of the average number of critical points at fixed value of the empirical risk. This result is simplified, and extended, using the non-rigorous Kac-Rice replicated method from theoretical physics. In this way we find an explicit variational formula for the quenched complexity, which is generally different from its annealed counterpart, and allows to obtain the number of critical points for typical instances up to exponential accuracy.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 3; 2023: 1; 2024: 4; 2025: 1; 2026: 2.

15 papers.

Sort:

  1. 2022Learning Single-Index Models with Shallow Neural Networkscited by 92
  2. 2021Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problemcited by 23
  3. 2021Local convexity of the TAP free energy and AMP convergence for Z2-synchronizationcited by 15
  4. 2022Learning a Single Neuron for Non-monotonic Activation Functionscited by 13
  5. 2024Exploring Neural Network Landscapes: Star-Shaped and Geodesic Connectivitycited by 9
  6. 2021Appearance of Random Matrix Theory in Deep Learningcited by 7
  7. 2024The Role of the Time-Dependent Hessian in High-Dimensional Optimizationcited by 7
  8. 2025Provable Benefits of Unsupervised Pre-training and Transfer Learning via Single-Index Modelscited by 6
  9. 2021A spin-glass model for the loss surfaces of generative adversarial networkscited by 4
  10. 2022Universal characteristics of deep neural network loss surfaces from random matrix theorycited by 4
  11. 2024On the number of modes of Gaussian kernel density estimatorscited by 4
  12. 2024Asymptotic Dynamics of Alternating Minimization for Bilinear Regressioncited by 3
3 more
  1. 2026Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networkscited by 2
  2. 2023Topological complexity of spiked random polynomials and finite-rank spherical integralscited by 0
  3. 2026Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrievalcited by 0