# A feature selection method based on Shapley values robust to concept shift in regression

arXiv: https://arxiv.org/abs/2304.14774 · HTML: https://mlsanity.com/paper/2304.14774/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 47 references that are in the map.
- First posted on arXiv 28 Apr 2023.

## Builds on (most cited first, 12 of 12)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [Explaining individual predictions when features are dependent: More accurate approximations to Shapley values](https://mlsanity.com/paper/1903.10464/) (2019)
- [Feature relevance quantification in explainable AI: A causal problem](https://mlsanity.com/paper/1910.13413/) (2019)
- [Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation](https://mlsanity.com/paper/1903.10992/) (2019)
- [Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark](https://mlsanity.com/paper/2008.08004/) (2020)
- [Feature Selection: A Data Perspective](https://mlsanity.com/paper/1601.07996/) (2016)
- [Forecasting: theory and practice](https://mlsanity.com/paper/2012.03854/) (2020)
- [Distributional neural networks for electricity price forecasting](https://mlsanity.com/paper/2207.02832/) (2022)
- [Powershap: A Power-full Shapley Feature Selection Method](https://mlsanity.com/paper/2206.08394/) (2022)

## Built on by (most cited first, 6 of 6)

- [A Comprehensive Study of Shapley Value in Data Analytics](https://mlsanity.com/paper/2412.01460/) (2024)
- [Energy-Based Model for Accurate Estimation of Shapley Values in Feature Attribution](https://mlsanity.com/paper/2404.01078/) (2024)
- [Shap-Select: Lightweight Feature Selection Using SHAP Values and Regression](https://mlsanity.com/paper/2410.06815/) (2024)
- [Statistical Inference and Learning for Shapley Additive Explanations (SHAP)](https://mlsanity.com/paper/2602.10532/) (2026)
- [An adaptive standardisation methodology for Day-Ahead electricity price forecasting](https://mlsanity.com/paper/2311.02610/) (2023)
- [Novel GPU Boruta algorithms for feature selection from high-dimensional data](https://mlsanity.com/paper/2605.09950/) (2026)

## Abstract (arXiv)

Feature selection is one of the most relevant processes in any methodology for creating a statistical learning model. Usually, existing algorithms establish some criterion to select the most influential variables, discarding those that do not contribute to the model with any relevant information. This methodology makes sense in a static situation where the joint distribution of the data does not vary over time. However, when dealing with real data, it is common to encounter the problem of the dataset shift and, specifically, changes in the relationships between variables (concept shift). In this case, the influence of a variable cannot be the only indicator of its quality as a regressor of the model, since the relationship learned in the training phase may not correspond to the current situation. In tackling this problem, our approach establishes a direct relationship between the Shapley values and prediction errors, operating at a more local level to effectively detect the individual biases introduced by each variable. The proposed methodology is evaluated through various examples, including synthetic scenarios mimicking sudden and incremental shift situations, as well as two real-world cases characterized by concept shifts. Additionally, we perform three analyses of standard situations to assess the algorithm's robustness in the absence of shifts. The results demonstrate that our proposed algorithm significantly outperforms state-of-the-art feature selection methods in concept shift scenarios, while matching the performance of existing methodologies in static situations.