# Embedded methods for feature selection in neural networks

arXiv: https://arxiv.org/abs/2010.05834 · HTML: https://mlsanity.com/paper/2010.05834/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2020.

## Built on by (most cited first, 2 of 2)

- [Combining data assimilation and machine learning to estimate parameters of a convective-scale model](https://mlsanity.com/paper/2109.02953/) (2021)
- [Embedded feature selection in LSTM networks with multi-objective evolutionary ensemble learning for time series forecasting](https://mlsanity.com/paper/2312.17517/) (2023)

## Abstract (arXiv)

The representational capacity of modern neural network architectures has made them a default choice in various applications with high dimensional feature sets. But these high dimensional and potentially noisy features combined with the black box models like neural networks negatively affect the interpretability, generalizability, and the training time of these models. Here, I propose two integrated approaches for feature selection that can be incorporated directly into the parameter learning. One of them involves adding a drop-in layer and performing sequential weight pruning. The other is a sensitivity-based approach. I benchmarked both the methods against Permutation Feature Importance (PFI) - a general-purpose feature ranking method and a random baseline. The suggested approaches turn out to be viable methods for feature selection, consistently outperform the baselines on the tested datasets - MNIST, ISOLET, and HAR. We can add them to any existing model with only a few lines of code.