# Feature Selection for Huge Data via Minipatch Learning

arXiv: https://arxiv.org/abs/2010.08529 · HTML: https://mlsanity.com/paper/2010.08529/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2020.

## Built on by (most cited first, 9 of 9)

- [To the Fairness Frontier and Beyond: Identifying, Quantifying, and Optimizing the Fairness-Accuracy Pareto Frontier](https://mlsanity.com/paper/2206.00074/) (2022)
- [iLOCO: Distribution-Free Inference for Feature Interactions](https://mlsanity.com/paper/2502.06661/) (2025)
- [Fast and Interpretable Consensus Clustering via Minipatch Learning](https://mlsanity.com/paper/2110.02388/) (2021)
- [Fast and Accurate Graph Learning for Huge Data via Minipatch Ensembles](https://mlsanity.com/paper/2110.12067/) (2021)
- [Top-k Feature Importance Ranking](https://mlsanity.com/paper/2509.15420/) (2025)
- [Fair MP-BOOST: Fair and Interpretable Minipatch Boosting](https://mlsanity.com/paper/2404.01521/) (2024)
- [Fair Feature Importance Scores via Feature Occlusion and Permutation](https://mlsanity.com/paper/2602.09196/) (2026)
- [Cluster LOCO: Feature Importance For Interpreting Clusters](https://mlsanity.com/paper/2606.14592/) (2026)
- [LOCO-AdaMP: Built-in LOCO Inference for Adaptive Minipatch Ensembles with Enhanced Prediction](https://mlsanity.com/paper/2609.36396/) (2026)

## Abstract (arXiv)

Feature selection often leads to increased model interpretability, faster computation, and improved model performance by discarding irrelevant or redundant features. While feature selection is a well-studied problem with many widely-used techniques, there are typically two key challenges: i) many existing approaches become computationally intractable in huge-data settings with millions of observations and features; and ii) the statistical accuracy of selected features degrades in high-noise, high-correlation settings, thus hindering reliable model interpretation. We tackle these problems by proposing Stable Minipatch Selection (STAMPS) and Adaptive STAMPS (AdaSTAMPS). These are meta-algorithms that build ensembles of selection events of base feature selectors trained on many tiny, (adaptively-chosen) random subsets of both the observations and features of the data, which we call minipatches. Our approaches are general and can be employed with a variety of existing feature selection strategies and machine learning techniques. In addition, we provide theoretical insights on STAMPS and empirically demonstrate that our approaches, especially AdaSTAMPS, dominate competing methods in terms of feature selection accuracy and computational time.