# Generative Principal Component Analysis arXiv: https://arxiv.org/abs/2203.09693 · HTML: https://mlsanity.com/paper/2203.09693/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Builds on 33 of its 79 references that are in the map. - First posted on arXiv 18 Mar 2022. ## Builds on (most cited first, 30 of 33) - [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017) - [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014) - [Compressed Sensing using Generative Models](https://mlsanity.com/paper/1703.03208/) (2017) - [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020) - [Phase Retrieval via Wirtinger Flow: Theory and Algorithms](https://mlsanity.com/paper/1407.1065/) (2014) - [Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks](https://mlsanity.com/paper/1810.03982/) (2018) - [Phase Retrieval Under a Generative Prior](https://mlsanity.com/paper/1807.04261/) (2018) - [Phase Retrieval using Alternating Minimization](https://mlsanity.com/paper/1306.0160/) (2013) - [Solving Linear Inverse Problems Using GAN Priors: An Algorithm with Provable Guarantees](https://mlsanity.com/paper/1802.08406/) (2018) - [Compressed Sensing with Deep Image Prior and Learned Regularization](https://mlsanity.com/paper/1806.06438/) (2018) - [Instance-Optimal Compressed Sensing via Posterior Sampling](https://mlsanity.com/paper/2106.11438/) (2021) - [Optimality and Sub-optimality of PCA I: Spiked Random Matrix Models](https://mlsanity.com/paper/1807.00891/) (2018) - [Modeling Sparse Deviations for Compressed Sensing using Generative Models](https://mlsanity.com/paper/1807.01442/) (2018) - [Robust Compressed Sensing using Generative Models](https://mlsanity.com/paper/2006.09461/) (2020) - [Algorithmic Guarantees for Inverse Imaging with Untrained Network Priors](https://mlsanity.com/paper/1906.08763/) (2019) - [Nonconvex phase synchronization](https://mlsanity.com/paper/1601.06114/) (2016) - [Statistical and computational trade-offs in estimation of sparse principal components](https://mlsanity.com/paper/1408.5369/) (2014) - [Information-Theoretic Lower Bounds for Compressive Sensing with Generative Models](https://mlsanity.com/paper/1908.10744/) (2019) - [Sparse PCA via Covariance Thresholding](https://mlsanity.com/paper/1311.5179/) (2013) - [Sample Complexity Bounds for 1-bit Compressive Sensing and Binary Stable Embeddings with Generative Priors](https://mlsanity.com/paper/2002.01697/) (2020) - [The spiked matrix model with generative priors](https://mlsanity.com/paper/1905.12385/) (2019) - [Exact asymptotics for phase retrieval and compressed sensing with random generative priors](https://mlsanity.com/paper/1912.02008/) (2019) - [Alternating Phase Projected Gradient Descent with Generative Priors for Solving Compressive Phase Retrieval](https://mlsanity.com/paper/1903.02707/) (2019) - [On the Estimation Performance and Convergence Rate of the Generalized Power Method for Phase Synchronization](https://mlsanity.com/paper/1603.00211/) (2016) - [The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences](https://mlsanity.com/paper/1609.05820/) (2016) - [The Generalized Lasso with Nonlinear Observations and Generative Priors](https://mlsanity.com/paper/2006.12415/) (2020) - [Provable Compressed Sensing with Generative Priors via Langevin Dynamics](https://mlsanity.com/paper/2102.12643/) (2021) - [Towards Sample-Optimal Compressive Phase Retrieval with Sparse and Generative Priors](https://mlsanity.com/paper/2106.15358/) (2021) - [Sparse PCA through Low-rank Approximations](https://mlsanity.com/paper/1303.0551/) (2013) - [Robust 1-bit Compressive Sensing with Partial Gaussian Circulant Matrices and Generative Priors](https://mlsanity.com/paper/2108.03570/) (2021) ## Built on by (most cited first, 12 of 12) - [DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2304.06648/) (2023) - [Theoretical Perspectives on Deep Learning Methods in Inverse Problems](https://mlsanity.com/paper/2206.14373/) (2022) - [Non-Iterative Recovery from Nonlinear Observations using Generative Models](https://mlsanity.com/paper/2205.15749/) (2022) - [Misspecified Phase Retrieval with Generative Priors](https://mlsanity.com/paper/2210.05571/) (2022) - [Integrating Intermediate Layer Optimization and Projected Gradient Descent for Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2505.20789/) (2025) - [Projected Gradient Descent Algorithms for Solving Nonlinear Inverse Problems with Generative Priors](https://mlsanity.com/paper/2209.10093/) (2022) - [Solving Quadratic Systems with Full-Rank Matrices Using Sparse or Generative Priors](https://mlsanity.com/paper/2309.09032/) (2023) - [Generalized Eigenvalue Problems with Generative Priors](https://mlsanity.com/paper/2411.01326/) (2024) - [Learning Single Index Models with Diffusion Priors](https://mlsanity.com/paper/2505.21135/) (2025) - [Do algorithms and barriers for sparse principal component analysis extend to other structured settings?](https://mlsanity.com/paper/2307.13535/) (2023) - [Diffusion Model Based Signal Recovery Under 1-Bit Quantization](https://mlsanity.com/paper/2511.12471/) (2025) - [Using Intermediate Forward Iterates for Intermediate Generator Optimization](https://mlsanity.com/paper/2302.02336/) (2023) ## Abstract (arXiv) In this paper, we study the problem of principal component analysis with generative modeling assumptions, adopting a general model for the observed matrix that encompasses notable special cases, including spiked matrix recovery and phase retrieval. The key assumption is that the underlying signal lies near the range of an L-Lipschitz continuous generative model with bounded k-dimensional inputs. We propose a quadratic estimator, and show that it enjoys a statistical rate of order √fracklog Lm, where m is the number of samples. We also provide a near-matching algorithm-independent lower bound. Moreover, we provide a variant of the classic power method, which projects the calculated data onto the range of the generative model during each iteration. We show that under suitable conditions, this method converges exponentially fast to a point achieving the above-mentioned statistical rate. We perform experiments on various image datasets for spiked matrix and phase retrieval models, and illustrate performance gains of our method to the classic power method and the truncated power method devised for sparse principal component analysis.