# Penalized Principal Component Analysis Using Smoothing

arXiv: https://arxiv.org/abs/2309.13838 · HTML: https://mlsanity.com/paper/2309.13838/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 47 references that are in the map.
- First posted on arXiv 25 Sep 2023.

## Builds on (most cited first, 6 of 6)

- [Sparse Generalized Eigenvalue Problem via Smooth Optimization](https://mlsanity.com/paper/1408.6686/) (2014)
- [Sparse Generalized Eigenvalue Problem: Optimal Statistical Rates via Truncated Rayleigh Flow](https://mlsanity.com/paper/1604.08697/) (2016)
- [Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem](https://mlsanity.com/paper/1711.03170/) (2017)
- [Penalized versus constrained generalized eigenvalue problems](https://mlsanity.com/paper/1410.6131/) (2014)
- [Sparse quadratic classification rules via linear dimension reduction](https://mlsanity.com/paper/1711.04817/) (2017)
- [ℓ1-norm constrained multi-block sparse canonical correlation analysis via proximal gradient descent](https://mlsanity.com/paper/2201.05289/) (2022)

## Abstract (arXiv)

Principal components computed via PCA (principal component analysis) are traditionally used to reduce dimensionality in genomic data or to correct for population stratification. In this paper, we explore the penalized eigenvalue problem (PEP) which reformulates the computation of the first eigenvector as an optimization problem and adds an L1 penalty constraint to enforce sparseness of the solution. The contribution of our article is threefold. First, we extend PEP by applying smoothing to the original LASSO-type L1 penalty. This allows one to compute analytical gradients which enable faster and more efficient minimization of the objective function associated with the optimization problem. Second, we demonstrate how higher order eigenvectors can be calculated with PEP using established results from singular value decomposition (SVD). Third, we present four experimental studies to demonstrate the usefulness of the smoothed penalized eigenvectors. Using data from the 1000 Genomes Project dataset, we empirically demonstrate that our proposed smoothed PEP allows one to increase numerical stability and obtain meaningful eigenvectors. We also employ the penalized eigenvector approach in two additional real data applications (computation of a polygenic risk score and clustering), demonstrating that exchanging the penalized eigenvectors for their smoothed counterparts can increase prediction accuracy in polygenic risk scores and enhance discernibility of clusterings. Moreover, we compare our proposed smoothed PEP to seven state-of-the-art algorithms for sparse PCA and evaluate the accuracy of the obtained eigenvectors, their support recovery, and their runtime.