# Feature Selection using Stochastic Gates

arXiv: https://arxiv.org/abs/1810.04247 · HTML: https://mlsanity.com/paper/1810.04247/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2018.

## Built on by (most cited first, 12 of 12)

- [A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization](https://mlsanity.com/paper/2111.02355/) (2021)
- [GANDALF: Gated Adaptive Network for Deep Automated Learning of Features](https://mlsanity.com/paper/2207.08548/) (2022)
- [Sequential Attention for Feature Selection](https://mlsanity.com/paper/2209.14881/) (2022)
- [Support Recovery with Stochastic Gates: Theory and Application for Linear Models](https://mlsanity.com/paper/2110.15960/) (2021)
- [Pruning-aware Sparse Regularization for Network Pruning](https://mlsanity.com/paper/2201.06776/) (2022)
- [DiSC: Differential Spectral Clustering of Features](https://mlsanity.com/paper/2211.05314/) (2022)
- [Learning to Increase the Power of Conditional Randomization Tests](https://mlsanity.com/paper/2207.01022/) (2022)
- [Enabling tabular deep learning when d gg n with an auxiliary knowledge graph](https://mlsanity.com/paper/2306.04766/) (2023)
- [Training Deep Models to be Explained with Fewer Examples](https://mlsanity.com/paper/2112.03508/) (2021)
- [Few-shot Learning for Unsupervised Feature Selection](https://mlsanity.com/paper/2107.00816/) (2021)
- [Enhanced Exploration in Neural Feature Selection for Deep Click-Through Rate Prediction Models via Ensemble of Gating Layers](https://mlsanity.com/paper/2112.03487/) (2021)
- [Sparse Centroid-Encoder: A Nonlinear Model for Feature Selection](https://mlsanity.com/paper/2201.12910/) (2022)

## Abstract (arXiv)

Feature selection problems have been extensively studied for linear estimation, for instance, Lasso, but less emphasis has been placed on feature selection for non-linear functions. In this study, we propose a method for feature selection in high-dimensional non-linear function estimation problems. The new procedure is based on minimizing the ℓ0 norm of the vector of indicator variables that represent if a feature is selected or not. Our approach relies on the continuous relaxation of Bernoulli distributions, which allows our model to learn the parameters of the approximate Bernoulli distributions via gradient descent. This general framework simultaneously minimizes a loss function while selecting relevant features. Furthermore, we provide an information-theoretic justification of incorporating Bernoulli distribution into our approach and demonstrate the potential of the approach on synthetic and real-life applications.