# Cluster Stability Selection

arXiv: https://arxiv.org/abs/2201.00494 · HTML: https://mlsanity.com/paper/2201.00494/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 66 references that are in the map.
- First posted on arXiv 3 Jan 2022.

## Builds on (most cited first, 8 of 8)

- [Veridical Data Science](https://mlsanity.com/paper/1901.08152/) (2019)
- [Adaptive Estimation in Structured Factor Models with Applications to Overlapping Clustering](https://mlsanity.com/paper/1704.06977/) (2017)
- [Gene Hunting with Knockoffs for Hidden Markov Models](https://mlsanity.com/paper/1706.04677/) (2017)
- [Graph-based regularization for regression problems with alignment and highly-correlated designs](https://mlsanity.com/paper/1803.07658/) (2018)
- [Sparse regression and marginal testing using cluster prototypes](https://mlsanity.com/paper/1503.00334/) (2015)
- [Pivotal Estimation via Self-Normalization for High-Dimensional Linear Models with Error in Variables](https://mlsanity.com/paper/1708.08353/) (2017)
- [Extensions of stability selection using subsamples of observations and covariates](https://mlsanity.com/paper/1407.4916/) (2014)
- [Simulation-Selection-Extrapolation: Estimation in High-Dimensional Errors-in-Variables Models](https://mlsanity.com/paper/1808.10477/) (2018)

## Built on by (most cited first, 4 of 4)

- [Stability Selection via Variable Decorrelation](https://mlsanity.com/paper/2505.20864/) (2025)
- [A Stable Lasso](https://mlsanity.com/paper/2511.02306/) (2025)
- [2D Stability Selection: Design Jittering for Doubly Stable Feature Selection](https://mlsanity.com/paper/2605.02205/) (2026)
- [SeqMaestro: From nucleotide sequences to biological hypotheses through interpretable machine learning](https://mlsanity.com/paper/2609.14882/) (2026)

## Abstract (arXiv)

Stability selection (Meinshausen and Buhlmann, 2010) makes any feature selection method more stable by returning only those features that are consistently selected across many subsamples. We prove (in what is, to our knowledge, the first result of its kind) that for data containing highly correlated proxies for an important latent variable, the lasso typically selects one proxy, yet stability selection with the lasso can fail to select any proxy, leading to worse predictive performance than the lasso alone. We introduce cluster stability selection, which exploits the practitioner's knowledge that highly correlated clusters exist in the data, resulting in better feature rankings than stability selection in this setting. We consider several feature-combination approaches, including taking a weighted average of the features in each important cluster where weights are determined by the frequency with which cluster members are selected, which we show leads to better predictive models than previous proposals. We present generalizations of theoretical guarantees from Meinshausen and Buhlmann (2010) and Shah and Samworth (2012) to show that cluster stability selection retains the same guarantees. In summary, cluster stability selection enjoys the best of both worlds, yielding a sparse selected set that is both stable and has good predictive performance.