# Feature Selection for Imbalanced Data with Deep Sparse Autoencoders Ensemble

arXiv: https://arxiv.org/abs/2103.11678 · HTML: https://mlsanity.com/paper/2103.11678/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability.
- Builds on 5 of its 58 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Placed in

- Mechanistic interpretability / [Sparse autoencoder training and evaluation](https://mlsanity.com/mechinterp/sparse-autoencoder-training-and-evaluation/)

## Builds on (most cited first, 5 of 5)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Concrete Autoencoders for Differentiable Feature Selection and Reconstruction](https://mlsanity.com/paper/1901.09346/) (2019)
- [Unsupervised Boosting-based Autoencoder Ensembles for Outlier Detection](https://mlsanity.com/paper/1910.09754/) (2019)
- [Cost-Sensitive Feature Selection by Optimizing F-Measures](https://mlsanity.com/paper/1904.02301/) (2019)
- [Graph Autoencoder-Based Unsupervised Feature Selection with Broad and Local Data Structure Preservation](https://mlsanity.com/paper/1801.02251/) (2018)

## Built on by (most cited first, 3 of 3)

- [Learning Signal Representations for EEG Cross-Subject Channel Selection and Trial Classification](https://mlsanity.com/paper/2106.10633/) (2021)
- [Distance Rank Score: Unsupervised filter method for feature selection on imbalanced dataset](https://mlsanity.com/paper/2305.19804/) (2023)
- [Mining of Single-Class by Active Learning for Semantic Segmentation](https://mlsanity.com/paper/2307.09109/) (2023)

## Abstract (arXiv)

Class imbalance is a common issue in many domain applications of learning algorithms. Oftentimes, in the same domains it is much more relevant to correctly classify and profile minority class observations. This need can be addressed by Feature Selection (FS), that offers several further advantages, s.a. decreasing computational costs, aiding inference and interpretability. However, traditional FS techniques may become sub-optimal in the presence of strongly imbalanced data. To achieve FS advantages in this setting, we propose a filtering FS algorithm ranking feature importance on the basis of the Reconstruction Error of a Deep Sparse AutoEncoders Ensemble (DSAEE). We use each DSAE trained only on majority class to reconstruct both classes. From the analysis of the aggregated Reconstruction Error, we determine the features where the minority class presents a different distribution of values w.r.t. the overrepresented one, thus identifying the most relevant features to discriminate between the two. We empirically demonstrate the efficacy of our algorithm in several experiments on high-dimensional datasets of varying sample size, showcasing its capability to select relevant and generalizable features to profile and classify minority class, outperforming other benchmark FS methods. We also briefly present a real application in radiogenomics, where the methodology was applied successfully.