# Empirical Bayes Covariance Decomposition, and a Solution to the Multiple Tuning Problem in Sparse PCA

arXiv: https://arxiv.org/abs/2312.03274 · HTML: https://mlsanity.com/paper/2312.03274/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 46 references that are in the map.
- First posted on arXiv 6 Dec 2023.

## Builds on (most cited first, 8 of 8)

- [Spectral Methods for Data Science: A Statistical Perspective](https://mlsanity.com/paper/2012.08496/) (2020)
- [Vintage Factor Analysis with Varimax Performs Statistical Inference](https://mlsanity.com/paper/2004.05387/) (2020)
- [Empirical Bayes PCA in high dimensions](https://mlsanity.com/paper/2012.11676/) (2020)
- [Empirical Bayes Matrix Factorization](https://mlsanity.com/paper/1802.06931/) (2018)
- [A flexible empirical Bayes approach to multiple linear regression and connections with penalized regression](https://mlsanity.com/paper/2208.10910/) (2022)
- [Binary Component Decomposition Part I: The Positive-Semidefinite Case](https://mlsanity.com/paper/1907.13603/) (2019)
- [ebnm: An R Package for Solving the Empirical Bayes Normal Means Problem Using a Variety of Prior Families](https://mlsanity.com/paper/2110.00152/) (2021)
- [An analysis of the cost of hyper-parameter selection via split-sample validation, with applications to penalized regression](https://mlsanity.com/paper/1903.12297/) (2019)

## Abstract (arXiv)

Sparse Principal Components Analysis (PCA) has been proposed as a way to improve both interpretability and reliability of PCA. However, use of sparse PCA in practice is hindered by the difficulty of tuning the multiple hyperparameters that control the sparsity of different PCs (the "multiple tuning problem", MTP). Here we present a solution to the MTP using Empirical Bayes methods. We first introduce a general formulation for penalized PCA of a data matrix X, which includes some existing sparse PCA methods as special cases. We show that this formulation also leads to a penalized decomposition of the covariance (or Gram) matrix, XTX. We introduce empirical Bayes versions of these penalized problems, in which the penalties are determined by prior distributions that are estimated from the data by maximum likelihood rather than cross-validation. The resulting "Empirical Bayes Covariance Decomposition" provides a principled and efficient solution to the MTP in sparse PCA, and one that can be immediately extended to incorporate other structural assumptions (e.g. non-negative PCA). We illustrate the effectiveness of this approach on both simulated and real data examples.