# Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders

arXiv: https://arxiv.org/abs/2012.00560 · HTML: https://mlsanity.com/paper/2012.00560/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2020.

## Built on by (most cited first, 28 of 28)

- [Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training](https://mlsanity.com/paper/2102.02887/) (2021)
- [Sparse Training via Boosting Pruning Plasticity with Neuroregeneration](https://mlsanity.com/paper/2106.10404/) (2021)
- [Head2Toe: Utilizing Intermediate Representations for Better Transfer Learning](https://mlsanity.com/paper/2201.03529/) (2022)
- [Dynamic Sparse Training for Deep Reinforcement Learning](https://mlsanity.com/paper/2106.04217/) (2021)
- [Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity](https://mlsanity.com/paper/2106.14568/) (2021)
- [Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers](https://mlsanity.com/paper/2302.02596/) (2023)
- [E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation](https://mlsanity.com/paper/2312.04727/) (2023)
- [Where to Pay Attention in Sparse Training for Feature Selection?](https://mlsanity.com/paper/2211.14627/) (2022)
- [Sparse Training Theory for Scalable and Efficient Agents](https://mlsanity.com/paper/2103.01636/) (2021)
- [Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning](https://mlsanity.com/paper/2302.06548/) (2023)
- [Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness](https://mlsanity.com/paper/2410.03030/) (2024)
- [Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networks](https://mlsanity.com/paper/2110.05329/) (2021)
- [Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance](https://mlsanity.com/paper/2203.02770/) (2022)
- [Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks](https://mlsanity.com/paper/2303.07200/) (2023)
- [UDRN: Unified Dimensional Reduction Neural Network for Feature Selection and Feature Projection](https://mlsanity.com/paper/2207.03809/) (2022)
- [Unveiling the Power of Sparse Neural Networks for Feature Selection](https://mlsanity.com/paper/2408.04583/) (2024)
- [Effective Neural Network L0 Regularization With BinMask](https://mlsanity.com/paper/2304.11237/) (2023)
- [Gradient-Congruity Guided Federated Sparse Training](https://mlsanity.com/paper/2405.01189/) (2024)
- [Leave it to the Specialist: Repair Sparse LLMs with Sparse Fine-Tuning via Sparsity Evolution](https://mlsanity.com/paper/2505.24037/) (2025)
- [Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers](https://mlsanity.com/paper/2305.18382/) (2023)
- [Knoop: Practical Enhancement of Knockoff with Over-Parameterization for Variable Selection](https://mlsanity.com/paper/2501.17889/) (2025)
- [Embedded Federated Feature Selection with Dynamic Sparse Training: Balancing Accuracy-Cost Tradeoffs](https://mlsanity.com/paper/2504.05245/) (2025)
- [Sparse-to-Sparse Training of Diffusion Models](https://mlsanity.com/paper/2504.21380/) (2025)
- [SAND: One-Shot Feature Selection with Additive Noise Distortion](https://mlsanity.com/paper/2505.03923/) (2025)
- [Model Input-Output Configuration Search with Embedded Feature Selection for Sensor Time-series and Image Classification](https://mlsanity.com/paper/2310.17250/) (2023)
- [EntryPrune: Neural Network Feature Selection using First Impressions](https://mlsanity.com/paper/2410.02344/) (2024)
- [A Topological Improvement of the Overall Performance of Sparse Evolutionary Training: Motif-Based Structural Optimization of Sparse MLPs Project](https://mlsanity.com/paper/2506.09204/) (2025)
- [Beyond Masked Sparsity: SNACK Enables Truly Sparse Neural Networks on GPU](https://mlsanity.com/paper/2610.04093/) (2026)

## Abstract (arXiv)

Major complications arise from the recent increase in the amount of high-dimensional data, including high computational costs and memory requirements. Feature selection, which identifies the most relevant and informative attributes of a dataset, has been introduced as a solution to this problem. Most of the existing feature selection methods are computationally inefficient; inefficient algorithms lead to high energy consumption, which is not desirable for devices with limited computational and energy resources. In this paper, a novel and flexible method for unsupervised feature selection is proposed. This method, named QuickSelection, introduces the strength of the neuron in sparse neural networks as a criterion to measure the feature importance. This criterion, blended with sparsely connected denoising autoencoders trained with the sparse evolutionary training procedure, derives the importance of all input features simultaneously. We implement QuickSelection in a purely sparse manner as opposed to the typical approach of using a binary mask over connections to simulate sparsity. It results in a considerable speed increase and memory reduction. When tested on several benchmark datasets, including five low-dimensional and three high-dimensional datasets, the proposed method is able to achieve the best trade-off of classification and clustering accuracy, running time, and maximum memory usage, among widely used approaches for feature selection. Besides, our proposed method requires the least amount of energy among the state-of-the-art autoencoder-based feature selection methods.