# Influential Feature PCA for high dimensional clustering

arXiv: https://arxiv.org/abs/1407.5241 · HTML: https://mlsanity.com/paper/1407.5241/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2014.

## Built on by (most cited first, 14 of 14)

- [Selective inference for k-means clustering](https://mlsanity.com/paper/2203.15267/) (2022)
- [Semi-Supervised Sparse Gaussian Classification: Provable Benefits of Unlabeled Data](https://mlsanity.com/paper/2409.03335/) (2024)
- [Sharp-SSL: Selective high-dimensional axis-aligned random projections for semi-supervised learning](https://mlsanity.com/paper/2304.09154/) (2023)
- [On the Use of Relative Validity Indices for Comparing Clustering Approaches](https://mlsanity.com/paper/2404.10351/) (2024)
- [On unsupervised projections and second order signals](https://mlsanity.com/paper/2204.05139/) (2022)
- [Subject clustering by IF-PCA and several recent methods](https://mlsanity.com/paper/2306.05363/) (2023)
- [Preprocessing power weighted shortest path data using a s-Well Separated Pair Decomposition](https://mlsanity.com/paper/2103.11216/) (2021)
- [Fitting networks with a cancellation trick](https://mlsanity.com/paper/2502.16728/) (2025)
- [High-dimensional Clustering and Signal Recovery under Block Signals](https://mlsanity.com/paper/2504.08332/) (2025)
- [GOLFS: Feature Selection via Combining Both Global and Local Information for High Dimensional Clustering](https://mlsanity.com/paper/2507.10956/) (2025)
- [i-IF-Learn: Iterative Feature Selection and Unsupervised Learning for High-Dimensional Complex Data](https://mlsanity.com/paper/2603.24025/) (2026)
- [Cluster LOCO: Feature Importance For Interpreting Clusters](https://mlsanity.com/paper/2606.14592/) (2026)
- [MNAR-k-means: A k-means Clustering for Data Missing Not at Random with Magnitude-Decaying Probability](https://mlsanity.com/paper/2606.31253/) (2026)
- [Selective Inference for Clustering Under Unknown Covariance](https://mlsanity.com/paper/2610.17458/) (2026)

## Abstract (arXiv)

We consider a clustering problem where we observe feature vectors Xi ∈ Rp, i = 1, 2, ldots, n, from K possible classes. The class labels are unknown and the main interest is to estimate them. We are primarily interested in the modern regime of p gg n, where classical clustering methods face challenges. We propose Influential Features PCA (IF-PCA) as a new clustering procedure. In IF-PCA, we select a small fraction of features with the largest Kolmogorov-Smirnov (KS) scores, where the threshold is chosen by adapting the recent notion of Higher Criticism, obtain the first (K-1) left singular vectors of the post-selection normalized data matrix, and then estimate the labels by applying the classical k-means to these singular vectors. It can be seen that IF-PCA is a tuning free clustering method. We apply IF-PCA to 10 gene microarray data sets. The method has competitive performance in clustering. Especially, in three of the data sets, the error rates of IF-PCA are only 29% or less of the error rates by other methods. We have also rediscovered a phenomenon on empirical null by citeEfron on microarray data. With delicate analysis, especially post-selection eigen-analysis, we derive tight probability bounds on the Kolmogorov-Smirnov statistics and show that IF-PCA yields clustering consistency in a broad context. The clustering problem is connected to the problems of sparse PCA and low-rank matrix recovery, but it is different in important ways. We reveal an interesting phase transition phenomenon associated with these problems and identify the range of interest for each.