# Local minima of the empirical risk in high dimension: General theorems and convex examples

arXiv: https://arxiv.org/abs/2502.01953 · HTML: https://mlsanity.com/paper/2502.01953/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2025.

## Built on by (most cited first, 4 of 4)

- [Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networks](https://mlsanity.com/paper/2604.09412/) (2026)
- [Landscape Complexity for the Empirical Risk of Generalized Linear Models: Discrimination between Structured Data](https://mlsanity.com/paper/2503.14403/) (2025)
- [Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrieval](https://mlsanity.com/paper/2602.17779/) (2026)
- [Influence Diagnostics in High-dimensional M-estimation: Precise Asymptotics](https://mlsanity.com/paper/2607.09250/) (2026)

## Abstract (arXiv)

We consider a general model for high-dimensional empirical risk minimization whereby the data xi are d-dimensional Gaussian vectors, the model is parametrized by Theta∈ℝd× k, and the loss depends on the data via the projection Theta^mathsfTxi. This setting covers as special cases classical statistics methods (e.g. multinomial regression and other generalized linear models), but also two-layer fully connected neural networks with k hidden neurons. We use the Kac-Rice formula from Gaussian process theory to derive a bound on the expected number of local minima of this empirical risk, under the proportional asymptotics in which n,d→∞, with nasymp d. Via Markov's inequality, this bound allows to determine the positions of these minimizers (with exponential deviation bounds) and hence derive sharp asymptotics on the estimation and prediction error. As a special case, we apply our characterization to convex losses. We show that our approach is tight and allows to prove previously conjectured results. In addition, we characterize the spectrum of the Hessian at the minimizer. A companion paper applies our general result to non-convex examples.