# Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem

arXiv: https://arxiv.org/abs/1711.03170 · HTML: https://mlsanity.com/paper/1711.03170/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2017.

## Built on by (most cited first, 4 of 4)

- [Generalized Eigenvalue Problems with Generative Priors](https://mlsanity.com/paper/2411.01326/) (2024)
- [Double-matched matrix decomposition for multi-view data](https://mlsanity.com/paper/2105.03396/) (2021)
- [ℓ1-norm constrained multi-block sparse canonical correlation analysis via proximal gradient descent](https://mlsanity.com/paper/2201.05289/) (2022)
- [Penalized Principal Component Analysis Using Smoothing](https://mlsanity.com/paper/2309.13838/) (2023)

## Abstract (arXiv)

We propose a new algorithm for sparse estimation of eigenvectors in generalized eigenvalue problems (GEP). The GEP arises in a number of modern data-analytic situations and statistical methods, including principal component analysis (PCA), multiclass linear discriminant analysis (LDA), canonical correlation analysis (CCA), sufficient dimension reduction (SDR) and invariant co-ordinate selection. We propose to modify the standard generalized orthogonal iteration with a sparsity-inducing penalty for the eigenvectors. To achieve this goal, we generalize the equation-solving step of orthogonal iteration to a penalized convex optimization problem. The resulting algorithm, called penalized orthogonal iteration, provides accurate estimation of the true eigenspace, when it is sparse. Also proposed is a computationally more efficient alternative, which works well for PCA and LDA problems. Numerical studies reveal that the proposed algorithms are competitive, and that our tuning procedure works well. We demonstrate applications of the proposed algorithm to obtain sparse estimates for PCA, multiclass LDA, CCA and SDR. Supplementary materials are available online.