# Variance-Preserving Orthogonal Selection (VPOS): Greedy Feature Selection via Orthogonal Deflation in PCA Loading Space

arXiv: https://arxiv.org/abs/2607.23198 · HTML: https://mlsanity.com/paper/2607.23198/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 35 references that are in the map.
- First posted on arXiv 25 Jul 2026.

## Builds on (most cited first, 3 of 3)

- [Gram-Schmidt Methods for Unsupervised Feature Extraction and Selection](https://mlsanity.com/paper/2311.09386/) (2023)
- [Enhancing Unsupervised Feature Selection via Double Sparsity Constrained Optimization](https://mlsanity.com/paper/2501.00726/) (2025)
- [An Interpretable and Stable Framework for Sparse Principal Component Analysis](https://mlsanity.com/paper/2603.13806/) (2026)

## Abstract (arXiv)

We present Variance-Preserving Orthogonal Selection (VPOS), an unsupervised feature-selection method that performs sequential orthogonal deflation in the variance-weighted principal component analysis (PCA) loading space VdLambdad1/2. After each feature is selected, its loading direction is projected out of all remaining candidates, so subsequent selections cover complementary directions of the rank-d covariance approximation while returning original variables. We establish rank-reduction guarantees and a determinant-growth interpretation, and distinguish VPOS from greedy selection on raw data, unweighted eigenvector pivoting, Principal Feature Analysis (PFA), and Principal Variable Selection (PVS). Experiments enforce k≤ d, tune method-specific parameters on validation observations, and evaluate on unseen outer folds. Across seven labelled benchmarks, VPOS improves held-out normalised reconstruction error over matched PCA without deflation on every dataset, with reductions of 1--78%. It obtains the lowest mean reconstruction error on Wine, Breast Cancer, and MNIST and is within 1.7% of the lowest error on CIFAR-10 and HighDim. On CIFAR-10, VPOS is approximately 24× faster than the closely related PVS baseline while incurring a 1.7% reconstruction gap. These results establish VPOS as an efficient covariance-coverage method, particularly when correlated high-dimensional data must be represented by a small set of identifiable original variables.