# Heteroskedastic PCA: Algorithm, Optimality, and Applications

arXiv: https://arxiv.org/abs/1810.08316 · HTML: https://mlsanity.com/paper/1810.08316/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2018.

## Built on by (most cited first, 30 of 34)

- [The Power of Contrast for Feature Learning: A Theoretical Analysis](https://mlsanity.com/paper/2110.02473/) (2021)
- [Understanding Multimodal Contrastive Learning and Incorporating Unpaired Data](https://mlsanity.com/paper/2302.06232/) (2023)
- [Inference for Heteroskedastic PCA with Missing Data](https://mlsanity.com/paper/2107.12365/) (2021)
- [Subspace Recovery from Heterogeneous Data with Non-isotropic Noise](https://mlsanity.com/paper/2210.13497/) (2022)
- [Optimal Differentially Private PCA and Estimation for Spiked Covariance Matrices](https://mlsanity.com/paper/2401.03820/) (2024)
- [Fundamental limits for rank-one matrix estimation with groupwise heteroskedasticity](https://mlsanity.com/paper/2106.11950/) (2021)
- [Estimating Higher-Order Mixed Memberships via the ℓ2,∞ Tensor Perturbation Bound](https://mlsanity.com/paper/2212.08642/) (2022)
- [Deflated HeteroPCA: Overcoming the curse of ill-conditioning in heteroskedastic PCA](https://mlsanity.com/paper/2303.06198/) (2023)
- [Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training](https://mlsanity.com/paper/2306.08173/) (2023)
- [Optimal Estimation of Shared Singular Subspaces across Multiple Noisy Matrices](https://mlsanity.com/paper/2411.17054/) (2024)
- [High-dimensional analysis of ridge regression for non-identically distributed data with a variance profile](https://mlsanity.com/paper/2403.20200/) (2024)
- [Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methods](https://mlsanity.com/paper/2405.13912/) (2024)
- [Self-Supervised Metric Learning in Multi-View Data: A Downstream Task Perspective](https://mlsanity.com/paper/2106.07138/) (2021)
- [Contrastive Learning on Multimodal Analysis of Electronic Health Records](https://mlsanity.com/paper/2403.14926/) (2024)
- [LEARNER: A Transfer Learning Method for Low-Rank Matrix Estimation](https://mlsanity.com/paper/2412.20605/) (2024)
- [Treatment Effect Estimation with Unobserved and Heterogeneous Confounding Variables](https://mlsanity.com/paper/2207.14439/) (2022)
- [ALPCAH: Sample-wise Heteroscedastic PCA with Tail Singular Value Regularization](https://mlsanity.com/paper/2307.02745/) (2023)
- [Optimal vintage factor analysis with deflation varimax](https://mlsanity.com/paper/2310.10545/) (2023)
- [ALPCAH: Subspace Learning for Sample-wise Heteroscedastic Data](https://mlsanity.com/paper/2505.07272/) (2025)
- [Minimax Estimation of Linear Functions of Eigenvectors in the Face of Small Eigen-Gaps](https://mlsanity.com/paper/2104.03298/) (2021)
- [Causal Discovery in High-Dimensional Point Process Networks with Hidden Nodes](https://mlsanity.com/paper/2109.10947/) (2021)
- [Uncertainty Quantification For Low-Rank Matrix Completion With Heterogeneous and Sub-Exponential Noise](https://mlsanity.com/paper/2110.12046/) (2021)
- [Mode-wise Principal Subspace Pursuit and Matrix Spiked Covariance Model](https://mlsanity.com/paper/2307.00575/) (2023)
- [Statistical Inference in Tensor Completion: Optimal Uncertainty Quantification and Statistical-to-Computational Gaps](https://mlsanity.com/paper/2410.11225/) (2024)
- [ALPCAHUS: Subspace Clustering for Heteroscedastic Data](https://mlsanity.com/paper/2505.18918/) (2025)
- [On Minimum Trace Factor Analysis -- An Old Song Sung to a New Tune](https://mlsanity.com/paper/2402.02459/) (2024)
- [Bias-Corrected Joint Spectral Embedding for Multilayer Networks with Invariant Subspace: Entrywise Eigenvector Perturbation and Inference](https://mlsanity.com/paper/2406.07849/) (2024)
- [Feature Selection for Latent Factor Models](https://mlsanity.com/paper/2412.10128/) (2024)
- [Subspace Recovery in Winsorized PCA: Insights into Accuracy and Robustness](https://mlsanity.com/paper/2502.16391/) (2025)
- [T-Rex: Fitting a Robust Factor Model via Expectation-Maximization](https://mlsanity.com/paper/2505.12117/) (2025)

## Abstract (arXiv)

A general framework for principal component analysis (PCA) in the presence of heteroskedastic noise is introduced. We propose an algorithm called HeteroPCA, which involves iteratively imputing the diagonal entries of the sample covariance matrix to remove estimation bias due to heteroskedasticity. This procedure is computationally efficient and provably optimal under the generalized spiked covariance model. A key technical step is a deterministic robust perturbation analysis on singular subspaces, which can be of independent interest. The effectiveness of the proposed algorithm is demonstrated in a suite of problems in high-dimensional statistics, including singular value decomposition (SVD) under heteroskedastic noise, Poisson PCA, and SVD for heteroskedastic and incomplete data.