# AFS: An Attention-based mechanism for Supervised Feature Selection

arXiv: https://arxiv.org/abs/1902.11074 · HTML: https://mlsanity.com/paper/1902.11074/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2019.

## Built on by (most cited first, 21 of 21)

- [Sequential Attention for Feature Selection](https://mlsanity.com/paper/2209.14881/) (2022)
- [TENT: Tensorized Encoder Transformer for Temperature Forecasting](https://mlsanity.com/paper/2106.14742/) (2021)
- [Higher-order Neural Additive Models: An Interpretable Machine Learning Model with Feature Interactions](https://mlsanity.com/paper/2209.15409/) (2022)
- [VisageSynTalk: Unseen Speaker Video-to-Speech Synthesis via Speech-Visage Feature Selection](https://mlsanity.com/paper/2206.07458/) (2022)
- [Graph Convolutional Network-based Feature Selection for High-dimensional and Low-sample Size Data](https://mlsanity.com/paper/2211.14144/) (2022)
- [FocusLearn: Fully-Interpretable, High-Performance Modular Neural Networks for Time Series](https://mlsanity.com/paper/2311.16834/) (2023)
- [Bridging Trustworthiness and Open-World Learning: An Exploratory Neural Approach for Enhancing Interpretability, Generalization, and Robustness](https://mlsanity.com/paper/2308.03666/) (2023)
- [Privileged Contrastive Pretraining for Multimodal Affect Modelling](https://mlsanity.com/paper/2508.03729/) (2025)
- [Variance Tolerance Factors For Interpreting ALL Neural Networks](https://mlsanity.com/paper/2209.13858/) (2022)
- [An Embedded Feature Selection Framework for Control](https://mlsanity.com/paper/2206.11064/) (2022)
- [Deep Feature Selection Using a Novel Complementary Feature Mask](https://mlsanity.com/paper/2209.12282/) (2022)
- [Embedded feature selection in LSTM networks with multi-objective evolutionary ensemble learning for time series forecasting](https://mlsanity.com/paper/2312.17517/) (2023)
- [The Origin of Self-Attention: Pairwise Affinity Matrices in Feature Selection and the Emergence of Self-Attention](https://mlsanity.com/paper/2507.14560/) (2025)
- [GRASP: group-Shapley feature selection for patients](https://mlsanity.com/paper/2602.11084/) (2026)
- [A-SFS: Semi-supervised Feature Selection based on Multi-task Self-supervision](https://mlsanity.com/paper/2207.09061/) (2022)
- [Experts in the Loop: Conditional Variable Selection for Accelerating Post-Silicon Analysis Based on Deep Learning](https://mlsanity.com/paper/2209.15249/) (2022)
- [Predicting Token Impact Towards Efficient Vision Transformer](https://mlsanity.com/paper/2305.14840/) (2023)
- [Deep Learning for Efficient GWAS Feature Selection](https://mlsanity.com/paper/2312.15055/) (2023)
- [Redundancy-optimized Multi-head Attention Networks for Multi-View Multi-Label Feature Selection](https://mlsanity.com/paper/2511.12462/) (2025)
- [MAFS: Multi-head Attention Feature Selection for High-Dimensional Data via Deep Fusion of Filter Methods](https://mlsanity.com/paper/2601.02668/) (2026)
- [Model-Agnostic FDR Control via Group Gaussian Mirror and Permutation SHAP](https://mlsanity.com/paper/2608.00989/) (2026)

## Abstract (arXiv)

As an effective data preprocessing step, feature selection has shown its effectiveness to prepare high-dimensional data for many machine learning tasks. The proliferation of high di-mension and huge volume big data, however, has brought major challenges, e.g. computation complexity and stability on noisy data, upon existing feature-selection techniques. This paper introduces a novel neural network-based feature selection architecture, dubbed Attention-based Feature Selec-tion (AFS). AFS consists of two detachable modules: an at-tention module for feature weight generation and a learning module for the problem modeling. The attention module for-mulates correlation problem among features and supervision target into a binary classification problem, supported by a shallow attention net for each feature. Feature weights are generated based on the distribution of respective feature se-lection patterns adjusted by backpropagation during the train-ing process. The detachable structure allows existing off-the-shelf models to be directly reused, which allows for much less training time, demands for the training data and requirements for expertise. A hybrid initialization method is also intro-duced to boost the selection accuracy for datasets without enough samples for feature weight generation. Experimental results show that AFS achieves the best accuracy and stability in comparison to several state-of-art feature selection algo-rithms upon both MNIST, noisy MNIST and several datasets with small samples.