# Shap-Select: Lightweight Feature Selection Using SHAP Values and Regression

arXiv: https://arxiv.org/abs/2410.06815 · HTML: https://mlsanity.com/paper/2410.06815/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 34 references that are in the map.
- First posted on arXiv 9 Oct 2024.

## Builds on (most cited first, 5 of 5)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Shapley values for feature selection: The good, the bad, and the axioms](https://mlsanity.com/paper/2102.10936/) (2021)
- [Powershap: A Power-full Shapley Feature Selection Method](https://mlsanity.com/paper/2206.08394/) (2022)
- [A feature selection method based on Shapley values robust to concept shift in regression](https://mlsanity.com/paper/2304.14774/) (2023)

## Built on by (most cited first, 1 of 1)

- [Statistical Inference and Learning for Shapley Additive Explanations (SHAP)](https://mlsanity.com/paper/2602.10532/) (2026)

## Abstract (arXiv)

Feature selection is an essential process in machine learning, especially when dealing with high-dimensional datasets. It helps reduce the complexity of machine learning models, improve performance, mitigate overfitting, and decrease computation time. This paper presents a novel feature selection framework, shap-select. The framework conducts a linear or logistic regression of the target on the Shapley values of the features, on the validation set, and uses the signs and significance levels of the regression coefficients to implement an efficient heuristic for feature selection in tabular regression and classification tasks. We evaluate shap-select on the Kaggle credit card fraud dataset, demonstrating its effectiveness compared to established methods such as Recursive Feature Elimination (RFE), HISEL (a mutual information-based feature selection method), Boruta and a simpler Shapley value-based method. Our findings show that shap-select combines interpretability, computational efficiency, and performance, offering a robust solution for feature selection.