mlsanityData to 14 Oct 2026

Spectral Methods for Data Science: A Statistical Perspective

arXiv 2012.08496 ↗First posted 15 Dec 2020stat.ML cs.IT cs.LGcited by 112

Not on the map. Why?

Abstract (arXiv)

Spectral methods have emerged as a simple yet surprisingly effective approach for extracting information from massive, noisy and incomplete data. In a nutshell, spectral methods refer to a collection of algorithms built upon the eigenvalues (resp. singular values) and eigenvectors (resp. singular vectors) of some properly designed matrices constructed from data. A diverse array of applications have been found in machine learning, data science, and signal processing. Due to their simplicity and effectiveness, spectral methods are not only used as a stand-alone estimator, but also frequently employed to initialize other more sophisticated algorithms to improve performance. While the studies of spectral methods can be traced back to classical matrix perturbation theory and methods of moments, the past decade has witnessed tremendous theoretical advances in demystifying their efficacy through the lens of statistical modeling, with the aid of non-asymptotic random matrix theory. This monograph aims to present a systematic, comprehensive, yet accessible introduction to spectral methods from a modern statistical perspective, highlighting their algorithmic implications in diverse large-scale applications. In particular, our exposition gravitates around several central questions that span various applications: how to characterize the sample efficiency of spectral methods in reaching a target level of statistical accuracy, and how to assess their stability in the face of random noise, missing data, and adversarial corruptions? In addition to conventional ℓ2 perturbation analysis, we present a systematic ℓ∞ and ℓ2,∞ perturbation theory for eigenspace and singular subspaces, which has only recently become available owing to a powerful "leave-one-out" analysis framework.

Key facts. Cited by 112 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 15; 2022: 22; 2023: 21; 2024: 20; 2025: 21; 2026: 13.

112 papers.

Sort:

By area:

  1. 2021Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstructioncited by 78
  2. 2023A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networkscited by 49
  3. 2023The Power of Preconditioning in Overparameterized Low-Rank Matrix Sensingcited by 33
  4. 2026Preconditioning Benefits of Spectral Orthogonalization in Muoncited by 30
  5. 2021Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurementscited by 28
  6. 2021Optimal Spectral Recovery of a Planted Vector in a Subspacecited by 21
  7. 2021Inference for Heteroskedastic PCA with Missing Datacited by 21
  8. 2022Uncertainty Quantification of MLE for Entity Ranking with Covariatescited by 16
  9. 2022Learning Mixtures of Linear Dynamical Systemscited by 15
  10. 2022Leave-one-out Singular Subspace Perturbation Analysis for Spectral Clusteringcited by 15
  11. 2022Generative modeling via tensor train sketchingcited by 13
  12. 2022Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomialscited by 13
38 more
  1. 2025Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorizationcited by 13
  2. 2023Spectral Entry-wise Matrix Estimation for Low-Rank Reinforcement Learningcited by 10
  3. 2024S2FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsitycited by 10
  4. 2023Approximate message passing from random initialization with applications to mathbbZ2 synchronizationcited by 9
  5. 2022Synthetically Controlled Banditscited by 8
  6. 2022Learning Low-Dimensional Nonlinear Structures from High-Dimensional Noisy Data: An Integral Operator Approachcited by 8
  7. 2022Estimating Higher-Order Mixed Memberships via the ℓ2,∞ Tensor Perturbation Boundcited by 8
  8. 2024LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learningcited by 8
  9. 2025Spectral Estimators for Multi-Index Models: Precise Asymptotics and Optimal Weak Recoverycited by 8
  10. 2021Partial recovery and weak consistency in the non-uniform hypergraph Stochastic Block Modelcited by 7
  11. 2023Deflated HeteroPCA: Overcoming the curse of ill-conditioning in heteroskedastic PCAcited by 7
  12. 2023Efficient Estimation of the Central Mean Subspace via Smoothed Gradient Outer Productscited by 7
  13. 2024Low-Tubal-Rank Tensor Recovery via Factorized Gradient Descentcited by 7
  14. 2024Statistical ranking with dynamic covariatescited by 6
  15. 2024Fast and Sample Efficient Multi-Task Representation Learning in Stochastic Contextual Banditscited by 6
  16. 2024Optimal Estimation of Shared Singular Subspaces across Multiple Noisy Matricescited by 6
  17. 2022Multiscale Graph Comparison via the Embedded Laplacian Discrepancycited by 5
  18. 2022One-Way Matching of Datasets with Low Rank Signalscited by 5
  19. 2022A New Central Limit Theorem for the Augmented IPW Estimator: Variance Inflation, Cross-Fit Covariance and Beyondcited by 5
  20. 2022Generative Modeling via Tree Tensor Network Statescited by 5
  21. 2023Fast and Accurate Estimation of Low-Rank Matrices from Noisy Measurements via Preconditioned Non-Convex Gradient Descentcited by 5
  22. 2023Intensity Profile Projection: A Framework for Continuous-Time Representation Learning for Dynamic Networkscited by 5
  23. 2023Spectral Estimators for Structured Generalized Linear Models via Approximate Message Passingcited by 5
  24. 2024A spectral method for multi-view subspace learning using the product of projectionscited by 5
  25. 2021Uncertainty-Guided Mixup for Semi-Supervised Domain Adaptation without Source Datacited by 4
  26. 2022Robust Matrix Completion with Heavy-tailed Noisecited by 4
  27. 2022Bipartite mixed membership distribution-free model. A novel model for community detection in overlapping bipartite weighted networkscited by 4
  28. 2025Demystifying Spectral Feature Learning for Instrumental Variable Regressioncited by 4
  29. 2025PCA++: How Uniformity Induces Robustness to Background Noise in Contrastive Learningcited by 4
  30. 2021Self-Supervised Metric Learning in Multi-View Data: A Downstream Task Perspectivecited by 3
  31. 2021A useful criterion on studying consistent estimation in community detectioncited by 3
  32. 2022Gradient Descent for Low-Rank Functionscited by 3
  33. 2023Near Optimal Heteroscedastic Regression with Symbiotic Learningcited by 3
  34. 2024Top-K ranking with a monotone adversarycited by 3
  35. 2024Non-convex matrix sensing: Breaking the quadratic rank barrier in the sample complexitycited by 3
  36. 2025From Invariant Representations to Invariant Data: Provable Robustness to Spurious Correlations via Noisy Counterfactual Matchingcited by 3
  37. 2021The Interplay of Demographic Variables and Social Distancing Scores in Deep Prediction of U.S. COVID-19 Casescited by 2
  38. 2021Directed mixed membership stochastic blockmodelcited by 2

50 most cited of 112 All 112 (JSON)