# Phase Transitions in Sparse PCA

arXiv: https://arxiv.org/abs/1503.00338 · HTML: https://mlsanity.com/paper/1503.00338/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Mar 2015.

## Built on by (most cited first, 15 of 15)

- [Graph-based Approximate Message Passing Iterations](https://mlsanity.com/paper/2109.11905/) (2021)
- [Hessian Eigenspectra of More Realistic Nonlinear Models](https://mlsanity.com/paper/2103.01519/) (2021)
- [The Franz-Parisi Criterion and Computational Trade-offs in High Dimensional Statistics](https://mlsanity.com/paper/2205.09727/) (2022)
- [Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective](https://mlsanity.com/paper/2308.14085/) (2023)
- [Local convexity of the TAP free energy and AMP convergence for Z2-synchronization](https://mlsanity.com/paper/2106.11428/) (2021)
- [Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealing](https://mlsanity.com/paper/2310.15440/) (2023)
- [Fundamental limits for rank-one matrix estimation with groupwise heteroskedasticity](https://mlsanity.com/paper/2106.11950/) (2021)
- [High-dimensional Asymptotics of VAEs: Threshold of Posterior Collapse and Dataset-Size Dependence of Rate-Distortion Curve](https://mlsanity.com/paper/2309.07663/) (2023)
- [Precise Error Rates for Computationally Efficient Testing](https://mlsanity.com/paper/2311.00289/) (2023)
- [Learning from higher-order statistics, efficiently: hypothesis tests, random features, and neural networks](https://mlsanity.com/paper/2312.14922/) (2023)
- [Bayesian reconstruction of memories stored in neural networks from their connectivity](https://mlsanity.com/paper/2105.07416/) (2021)
- [Automatic Hyperparameter Tuning in Sparse Matrix Factorization](https://mlsanity.com/paper/2305.10114/) (2023)
- [Statistical Mechanics of Min-Max Problems](https://mlsanity.com/paper/2409.06053/) (2024)
- [Orthogonal Approximate Message Passing with Optimal Spectral Initializations for Rectangular Spiked Matrix Models](https://mlsanity.com/paper/2512.19334/) (2025)
- [Orthogonal Approximate Message Passing Algorithms for Rectangular Spiked Matrix Models with Rotationally Invariant Noise](https://mlsanity.com/paper/2602.03283/) (2026)

## Abstract (arXiv)

We study optimal estimation for sparse principal component analysis when the number of non-zero elements is small but on the same order as the dimension of the data. We employ approximate message passing (AMP) algorithm and its state evolution to analyze what is the information theoretically minimal mean-squared error and the one achieved by AMP in the limit of large sizes. For a special case of rank one and large enough density of non-zeros Deshpande and Montanari [1] proved that AMP is asymptotically optimal. We show that both for low density and for large rank the problem undergoes a series of phase transitions suggesting existence of a region of parameters where estimation is information theoretically possible, but AMP (and presumably every other polynomial algorithm) fails. The analysis of the large rank limit is particularly instructive.