# Feature Selection using Sparse Adaptive Bottleneck Centroid-Encoder

arXiv: https://arxiv.org/abs/2306.04795 · HTML: https://mlsanity.com/paper/2306.04795/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 53 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Concrete Autoencoders for Differentiable Feature Selection and Reconstruction](https://mlsanity.com/paper/1901.09346/) (2019)
- [Group Sparse Regularization for Deep Neural Networks](https://mlsanity.com/paper/1607.00485/) (2016)
- [LassoNet: A Neural Network with Feature Sparsity](https://mlsanity.com/paper/1907.12207/) (2019)
- [A Review of Feature Selection Methods Based on Mutual Information](https://mlsanity.com/paper/1509.07577/) (2015)
- [FsNet: Feature Selection Network on High-dimensional Biological Data](https://mlsanity.com/paper/2001.08322/) (2020)
- [Supervised Dimensionality Reduction and Visualization using Centroid-encoder](https://mlsanity.com/paper/2002.11934/) (2020)
- [The Trimmed Lasso: Sparsity and Robustness](https://mlsanity.com/paper/1708.04527/) (2017)
- [The Efficacy of L1 Regularization in Two-Layer Neural Networks](https://mlsanity.com/paper/2010.01048/) (2020)

## Abstract (arXiv)

We introduce a novel nonlinear model, Sparse Adaptive Bottleneck Centroid-Encoder (SABCE), for determining the features that discriminate between two or more classes. The algorithm aims to extract discriminatory features in groups while reconstructing the class centroids in the ambient space and simultaneously use additional penalty terms in the bottleneck layer to decrease within-class scatter and increase the separation of different class centroids. The model has a sparsity-promoting layer (SPL) with a one-to-one connection to the input layer. Along with the primary objective, we minimize the l2,1-norm of the sparse layer, which filters out unnecessary features from input data. During training, we update class centroids by taking the Hadamard product of the centroids and weights of the sparse layer, thus ignoring the irrelevant features from the target. Therefore the proposed method learns to reconstruct the critical components of class centroids rather than the whole centroids. The algorithm is applied to various real-world data sets, including high-dimensional biological, image, speech, and accelerometer sensor data. We compared our method to different state-of-the-art feature selection techniques, including supervised Concrete Autoencoders (SCAE), Feature Selection Networks (FsNet), Stochastic Gates (STG), and LassoNet. We empirically showed that SABCE features often produced better classification accuracy than other methods on the sequester test sets, setting new state-of-the-art results.