# EntryPrune: Neural Network Feature Selection using First Impressions

arXiv: https://arxiv.org/abs/2410.02344 · HTML: https://mlsanity.com/paper/2410.02344/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Builds on (most cited first, 22 of 22)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [SNIP: Single-shot Network Pruning based on Connection Sensitivity](https://mlsanity.com/paper/1810.02340/) (2018)
- [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [Rigging the Lottery: Making All Tickets Winners](https://mlsanity.com/paper/1911.11134/) (2019)
- [Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science](https://mlsanity.com/paper/1707.04780/) (2017)
- [The Convergence of Sparsified Gradient Methods](https://mlsanity.com/paper/1809.10505/) (2018)
- [Sparse GPU Kernels for Deep Learning](https://mlsanity.com/paper/2006.10901/) (2020)
- [LassoNet: A Neural Network with Feature Sparsity](https://mlsanity.com/paper/1907.12207/) (2019)
- [Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders](https://mlsanity.com/paper/2012.00560/) (2020)
- [A Performance-Driven Benchmark for Feature Selection in Tabular Deep Learning](https://mlsanity.com/paper/2311.05877/) (2023)
- [Fantastic Weights and How to Find Them: Where to Prune in Dynamic Sparse Training](https://mlsanity.com/paper/2306.12230/) (2023)
- [Where to Pay Attention in Sparse Training for Feature Selection?](https://mlsanity.com/paper/2211.14627/) (2022)
- [Composite Feature Selection using Deep Ensembles](https://mlsanity.com/paper/2211.00631/) (2022)
- [Review of Swarm Intelligence-based Feature Selection Methods](https://mlsanity.com/paper/2008.04103/) (2020)
- [High Performance Unstructured SpMM Computation Using Tensor Cores](https://mlsanity.com/paper/2408.11551/) (2024)
- [Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks](https://mlsanity.com/paper/2303.07200/) (2023)
- [Unveiling the Power of Sparse Neural Networks for Feature Selection](https://mlsanity.com/paper/2408.04583/) (2024)

## Abstract (arXiv)

There is an ongoing effort to develop feature selection algorithms to improve interpretability, reduce computational resources, and minimize overfitting in predictive models. Neural networks stand out as architectures on which to build feature selection methods, and recently, neuron pruning and regrowth have emerged from the sparse neural network literature as promising new tools. We introduce EntryPrune, a novel supervised feature selection algorithm using a dense neural network with a dynamic sparse input layer. It employs entry-based pruning, a novel approach that compares neurons based on their relative change induced when they have entered the network. Extensive experiments on 13 different datasets show that our approach generally outperforms the current state-of-the-art methods, and in particular improves the average accuracy on low-dimensional datasets. Furthermore, we show that EntryPruning surpasses traditional techniques such as magnitude pruning within the EntryPrune framework and that EntryPrune achieves lower runtime than competing approaches. Our code is available at https://github.com/flxzimmer/entryprune.