# Penalized versus constrained generalized eigenvalue problems

arXiv: https://arxiv.org/abs/1410.6131 · HTML: https://mlsanity.com/paper/1410.6131/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2014.

## Built on by (most cited first, 3 of 3)

- [ℓ1-norm constrained multi-block sparse canonical correlation analysis via proximal gradient descent](https://mlsanity.com/paper/2201.05289/) (2022)
- [AgFlow: Fast Model Selection of Penalized PCA via Implicit Regularization Effects of Gradient Flow](https://mlsanity.com/paper/2110.03273/) (2021)
- [Penalized Principal Component Analysis Using Smoothing](https://mlsanity.com/paper/2309.13838/) (2023)

## Abstract (arXiv)

We investigate the difference between using an ℓ1 penalty versus an ℓ1 constraint in generalized eigenvalue problems, such as principal component analysis and discriminant analysis. Our main finding is that an ℓ1 penalty may fail to provide very sparse solutions; a severe disadvantage for variable selection that can be remedied by using an ℓ1 constraint. Our claims are supported both by empirical evidence and theoretical analysis. Finally, we illustrate the advantages of an ℓ1 constraint in the context of discriminant analysis and principal component analysis.