# Feature Gradients: Scalable Feature Selection via Discrete Relaxation

arXiv: https://arxiv.org/abs/1908.10382 · HTML: https://mlsanity.com/paper/1908.10382/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2019.

## Built on by (most cited first, 2 of 2)

- [A Data-driven feature selection and machine-learning model benchmark for the prediction of longitudinal dispersion coefficient](https://mlsanity.com/paper/2107.12970/) (2021)
- [Sensitivity Analysis On Loss Landscape](https://mlsanity.com/paper/2403.01128/) (2024)

## Abstract (arXiv)

In this paper we introduce Feature Gradients, a gradient-based search algorithm for feature selection. Our approach extends a recent result on the estimation of learnability in the sublinear data regime by showing that the calculation can be performed iteratively (i.e., in mini-batches) and in linear time and space with respect to both the number of features D and the sample size N . This, along with a discrete-to-continuous relaxation of the search domain, allows for an efficient, gradient-based search algorithm among feature subsets for very large datasets. Crucially, our algorithm is capable of finding higher-order correlations between features and targets for both the N > D and N < D regimes, as opposed to approaches that do not consider such interactions and/or only consider one regime. We provide experimental demonstration of the algorithm in small and large sample-and feature-size settings.