# SAND: One-Shot Feature Selection with Additive Noise Distortion

arXiv: https://arxiv.org/abs/2505.03923 · HTML: https://mlsanity.com/paper/2505.03923/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 6 May 2025.

## Builds on (most cited first, 12 of 12)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Concrete Autoencoders for Differentiable Feature Selection and Reconstruction](https://mlsanity.com/paper/1901.09346/) (2019)
- [Group Sparse Regularization for Deep Neural Networks](https://mlsanity.com/paper/1607.00485/) (2016)
- [Training Sparse Neural Networks](https://mlsanity.com/paper/1611.06694/) (2016)
- [Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders](https://mlsanity.com/paper/2012.00560/) (2020)
- [Where to Pay Attention in Sparse Training for Feature Selection?](https://mlsanity.com/paper/2211.14627/) (2022)
- [Feature Selection Using Batch-Wise Attenuation and Feature Mask Normalization](https://mlsanity.com/paper/2010.13631/) (2020)
- [Sequential Attention for Feature Selection](https://mlsanity.com/paper/2209.14881/) (2022)
- [Attentional Neural Network: Feature Selection Using Cognitive Feedback](https://mlsanity.com/paper/1411.5140/) (2014)
- [A scalable saliency-based Feature selection method with instance level information](https://mlsanity.com/paper/1904.13127/) (2019)
- [E2E-FS: An End-to-End Feature Selection Method for Neural Networks](https://mlsanity.com/paper/2012.07671/) (2020)

## Built on by (most cited first, 1 of 1)

- [PACE: Plug-and-Play Contextual Embedding for Feature Screening with Pretrained Tabular Foundation Models](https://mlsanity.com/paper/2609.23574/) (2026)

## Abstract (arXiv)

Feature selection is a critical step in data-driven applications, reducing input dimensionality to enhance learning accuracy, computational efficiency, and interpretability. Existing state-of-the-art methods often require post-selection retraining and extensive hyperparameter tuning, complicating their adoption. We introduce a novel, non-intrusive feature selection layer that, given a target feature count k, automatically identifies and selects the k most informative features during neural network training. Our method is uniquely simple, requiring no alterations to the loss function, network architecture, or post-selection retraining. The layer is mathematically elegant and can be fully described by: beginalign nonumber tildexi = ai xi + (1-ai)zi endalign where xi is the input feature, tildexi the output, zi a Gaussian noise, and ai trainable gain such that sumiai2=k. This formulation induces an automatic clustering effect, driving k of the ai gains to 1 (selecting informative features) and the rest to 0 (discarding redundant ones) via weighted noise distortion and gain normalization. Despite its extreme simplicity, our method delivers state-of-the-art performance on standard benchmark datasets and a novel real-world dataset, outperforming or matching existing approaches without requiring hyperparameter search for k or retraining. Theoretical analysis in the context of linear regression further validates its efficacy. Our work demonstrates that simplicity and performance are not mutually exclusive, offering a powerful yet straightforward tool for feature selection in machine learning.