# Decorrelated Variable Importance

arXiv: https://arxiv.org/abs/2111.10853 · HTML: https://mlsanity.com/paper/2111.10853/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 21 Nov 2021.

## Builds on (most cited first, 14 of 14)

- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [Explaining individual predictions when features are dependent: More accurate approximations to Shapley values](https://mlsanity.com/paper/1903.10464/) (2019)
- [Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regression](https://mlsanity.com/paper/2012.01536/) (2020)
- [Efficient nonparametric statistical inference on population feature importance using Shapley values](https://mlsanity.com/paper/2006.09481/) (2020)
- [Explaining black box decisions by Shapley cohort refinement](https://mlsanity.com/paper/1911.00467/) (2019)
- [SHAFF: Fast and consistent SHApley eFfect estimates via random Forests](https://mlsanity.com/paper/2105.11724/) (2021)
- [An unexpected unity among methods for interpreting model predictions](https://mlsanity.com/paper/1611.07478/) (2016)
- [Floodgate: inference for model-free variable importance](https://mlsanity.com/paper/2007.01283/) (2020)
- [Significance tests of feature relevance for a black-box learner](https://mlsanity.com/paper/2103.04985/) (2021)
- [Robust Estimation of Causal Effects via High-Dimensional Covariate Balancing Propensity Score](https://mlsanity.com/paper/1812.08683/) (2018)
- [A Semiparametric Approach to Interpretable Machine Learning](https://mlsanity.com/paper/2006.04732/) (2020)
- [Variable importance scores](https://mlsanity.com/paper/2102.07765/) (2021)
- [Shapley value confidence intervals for attributing variance explained](https://mlsanity.com/paper/2001.09593/) (2020)

## Built on by (most cited first, 13 of 13)

- [Feature Importance: A Closer Look at Shapley Values and LOCO](https://mlsanity.com/paper/2303.05981/) (2023)
- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [Rethinking Explainable Machine Learning as Applied Statistics](https://mlsanity.com/paper/2402.02870/) (2024)
- [SHAP-XRT: The Shapley Value Meets Conditional Independence Testing](https://mlsanity.com/paper/2207.07038/) (2022)
- [I Bet You Did Not Mean That: Testing Semantic Importance via Betting](https://mlsanity.com/paper/2405.19146/) (2024)
- [iLOCO: Distribution-Free Inference for Feature Interactions](https://mlsanity.com/paper/2502.06661/) (2025)
- [Shapley Curves: A Smoothing Perspective](https://mlsanity.com/paper/2211.13289/) (2022)
- [Conditional Mean and Variance Estimation via textitk-NN Algorithm with Automated Variance Selection](https://mlsanity.com/paper/2402.01635/) (2024)
- [Decorrelated feature importance from local sample weighting](https://mlsanity.com/paper/2508.06337/) (2025)
- [MinShap: A Shapley-Based Framework for Feature Redundancy](https://mlsanity.com/paper/2604.15107/) (2026)
- [Targeted Learning for Variable Importance](https://mlsanity.com/paper/2411.02221/) (2024)
- [AICO: Feature Significance Tests for Supervised Learning](https://mlsanity.com/paper/2506.23396/) (2025)
- [Cross-Fitting Under Nonregularity: Normality and Inference via Locality](https://mlsanity.com/paper/2610.02944/) (2026)

## Abstract (arXiv)

Because of the widespread use of black box prediction methods such as random forests and neural nets, there is renewed interest in developing methods for quantifying variable importance as part of the broader goal of interpretable prediction. A popular approach is to define a variable importance parameter - known as LOCO (Leave Out COvariates) - based on dropping covariates from a regression model. This is essentially a nonparametric version of R-squared. This parameter is very general and can be estimated nonparametrically, but it can be hard to interpret because it is affected by correlation between covariates. We propose a method for mitigating the effect of correlation by defining a modified version of LOCO. This new parameter is difficult to estimate nonparametrically, but we show how to estimate it using semiparametric models.