mlsanityData to 7 Oct 2026

Spectral Estimators for Multi-Index Models: Precise Asymptotics and Optimal Weak Recovery

arXiv 2502.01583 ↗First posted 3 Feb 2025stat.ML cs.IT cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Multi-index models provide a popular framework to investigate the learnability of functions with low-dimensional structure and, also due to their connections with neural networks, they have been object of recent intensive study. In this paper, we focus on recovering the subspace spanned by the signals via spectral estimators -- a family of methods routinely used in practice, often as a warm-start for iterative algorithms. Our main technical contribution is a precise asymptotic characterization of the performance of spectral methods, when sample size and input dimension grow proportionally and the dimension p of the space to recover is fixed. Specifically, we locate the top-p eigenvalues of the spectral matrix and establish the overlaps between the corresponding eigenvectors (which give the spectral estimators) and a basis of the signal subspace. Our analysis unveils a phase transition phenomenon in which, as the sample complexity grows, eigenvalues escape from the bulk of the spectrum and, when that happens, eigenvectors recover directions of the desired subspace. The precise characterization we put forward enables the optimization of the data preprocessing, thus allowing to identify the spectral estimator that requires the minimal sample size for weak recovery.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 26 of its 53 references that are in the map. First posted on arXiv 3 Feb 2025.

Builds on

26 of 53 references.

References · 26

  1. 2017Community Detection and Stochastic Block Modelscited by 317
  2. 2022Neural Networks can Learn Representations with Gradient Descentcited by 161
  3. 2023SGD learning on neural networks: leap complexity and saddle-to-saddle dynamicscited by 142
  4. 2020Online stochastic gradient descent on non-convex losses from high-dimensional inferencecited by 119
  5. 2017Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Modelscited by 115
  6. 2020Spectral Methods for Data Science: A Statistical Perspectivecited by 111
  7. 2014A statistical model for tensor PCAcited by 72
  8. 2023Smoothing the Landscape Boosts the Signal for SGD: Optimal Sample Complexity for Learning Single Index Modelscited by 60
  9. 2018The committee machine: Computational to statistical gaps in learning a two-layers neural networkcited by 58
  10. 2023On Learning Gaussian Multi-index Models with Gradient Flowcited by 53
  11. 2015Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systemscited by 50
  12. 2017Estimation of Low-Rank Matrices via Approximate Message Passingcited by 41
14 more
  1. 2017Fundamental Limits of Weak Recovery with Applications to Phase Retrievalcited by 39
  2. 2020Phase retrieval in high dimensions: Statistical and computational phase transitionscited by 39
  3. 2024Computational-Statistical Gaps in Gaussian Single-Index Modelscited by 39
  4. 2020Learning Polynomials of Few Relevant Dimensionscited by 34
  5. 2020The estimation error of general first order methodscited by 33
  6. 2016Solving Systems of Random Quadratic Equations via Truncated Amplitude Flowcited by 28
  7. 2017Phase Transitions of Spectral Initialization for High-Dimensional Nonconvex Estimationcited by 26
  8. 2023Hitting the High-Dimensional Notes: An ODE for SGD learning dynamics on GLMs and multi-index modelscited by 25
  9. 2020Approximate Message Passing with Spectral Initialization for Generalized Linear Modelscited by 22
  10. 2021Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passingcited by 18
  11. 2020Optimal Combination of Linear and Spectral Estimators for Generalized Linear Modelscited by 15
  12. 2024Learning sum of diverse features: computational hardness and efficient gradient-based training for ridge combinationscited by 13
  13. 2023Spectral Estimators for Structured Generalized Linear Models via Approximate Message Passingcited by 5
  14. 2024Learning Orthogonal Multi-Index Models: A Fine-Grained Information Exponent Analysiscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025Survey on Algorithms for multi-index modelscited by 15
  2. 2025Optimal Spectral Transitions in High-Dimensional Multi-Index Modelscited by 6
  3. 2025The Generative Leap: Sharp Sample Complexity for Efficiently Learning Gaussian Multi-Index Modelscited by 6
  4. 2026Optimal scaling laws in learning hierarchical multi-index modelscited by 6
  5. 2026Deep Learning of Compositional Targets with Hierarchical Spectral Methodscited by 2
  6. 2026A Noise Sensitivity Exponent Controls Large Statistical-to-Computational Gaps in Single- and Multi-Index Modelscited by 1