# Spike and slab Bayesian sparse principal component analysis

arXiv: https://arxiv.org/abs/2102.00305 · HTML: https://mlsanity.com/paper/2102.00305/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 60 references that are in the map.
- First posted on arXiv 30 Jan 2021.

## Builds on (most cited first, 15 of 15)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Frequentist Consistency of Variational Bayes](https://mlsanity.com/paper/1705.03439/) (2017)
- [Convergence Rates of Variational Posterior Distributions](https://mlsanity.com/paper/1712.02519/) (2017)
- [Variational Bayes for high-dimensional linear regression with sparse priors](https://mlsanity.com/paper/1904.07150/) (2019)
- [Bayesian linear regression with sparse priors](https://mlsanity.com/paper/1403.0735/) (2014)
- [Multivariate Bayesian Structural Time Series Model](https://mlsanity.com/paper/1801.03222/) (2018)
- [Empirical Bayes posterior concentration in sparse high-dimensional linear models](https://mlsanity.com/paper/1406.7718/) (2014)
- [Sparse Principal Component Analysis via Variable Projection](https://mlsanity.com/paper/1804.00341/) (2018)
- [A Variational Algorithm for Bayesian Variable Selection](https://mlsanity.com/paper/1602.07640/) (2016)
- [Bayesian method for causal inference in spatially-correlated multivariate time series](https://mlsanity.com/paper/1801.06282/) (2018)
- [VarFA: A Variational Factor Analysis Framework For Efficient Bayesian Learning Analytics](https://mlsanity.com/paper/2005.13107/) (2020)
- [Bayesian Estimation of Sparse Spiked Covariance Matrices in High Dimensions](https://mlsanity.com/paper/1808.07433/) (2018)
- [Bayesian Variable Selection for Globally Sparse Probabilistic PCA](https://mlsanity.com/paper/1605.05918/) (2016)
- [Heterogeneous large datasets integration using Bayesian factor regression](https://mlsanity.com/paper/1810.09894/) (2018)
- [Bayesian Feature Selection in Joint Quantile Time Series Analysis](https://mlsanity.com/paper/2010.01654/) (2020)

## Built on by (most cited first, 3 of 3)

- [Machine Learning and the Future of Bayesian Computation](https://mlsanity.com/paper/2304.11251/) (2023)
- [Variational Bayes for high-dimensional proportional hazards models with applications within gene expression](https://mlsanity.com/paper/2112.10270/) (2021)
- [A Bayesian sparse factor model with adaptive posterior concentration](https://mlsanity.com/paper/2305.18488/) (2023)

## Abstract (arXiv)

Sparse principal component analysis (SPCA) is a popular tool for dimensionality reduction in high-dimensional data. However, there is still a lack of theoretically justified Bayesian SPCA methods that can scale well computationally. One of the major challenges in Bayesian SPCA is selecting an appropriate prior for the loadings matrix, considering that principal components are mutually orthogonal. We propose a novel parameter-expanded coordinate ascent variational inference (PX-CAVI) algorithm. This algorithm utilizes a spike and slab prior, which incorporates parameter expansion to cope with the orthogonality constraint. Besides comparing to two popular SPCA approaches, we introduce the PX-EM algorithm as an EM analogue to the PX-CAVI algorithm for comparison. Through extensive numerical simulations, we demonstrate that the PX-CAVI algorithm outperforms these SPCA approaches, showcasing its superiority in terms of performance. We study the posterior contraction rate of the variational posterior, providing a novel contribution to the existing literature. The PX-CAVI algorithm is then applied to study a lung cancer gene expression dataset. The R package VBsparsePCA with an implementation of the algorithm is available on the Comprehensive R Archive Network (CRAN).