# Efficient nonparametric statistical inference on population feature importance using Shapley values

arXiv: https://arxiv.org/abs/2006.09481 · HTML: https://mlsanity.com/paper/2006.09481/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2020.

## Built on by (most cited first, 30 of 61)

- [FastSHAP: Real-Time Shapley Value Estimation](https://mlsanity.com/paper/2107.07436/) (2021)
- [Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities](https://mlsanity.com/paper/2111.06420/) (2021)
- [Algorithms to estimate Shapley value feature attributions](https://mlsanity.com/paper/2207.07605/) (2022)
- [The Shapley Value in Machine Learning](https://mlsanity.com/paper/2202.05594/) (2022)
- [SHAP-IQ: Unified Approximation of any-order Shapley Interactions](https://mlsanity.com/paper/2303.01179/) (2023)
- [Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experiments](https://mlsanity.com/paper/2206.10013/) (2022)
- [Explaining a Series of Models by Propagating Shapley Values](https://mlsanity.com/paper/2105.00108/) (2021)
- [Explaining Predictive Uncertainty with Information Theoretic Shapley Values](https://mlsanity.com/paper/2306.05724/) (2023)
- [The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importance](https://mlsanity.com/paper/2309.13775/) (2023)
- [Optimal ablation for interpretability](https://mlsanity.com/paper/2409.09951/) (2024)
- [The Shapley Value of Classifiers in Ensemble Games](https://mlsanity.com/paper/2101.02153/) (2021)
- [Feature Importance: A Closer Look at Shapley Values and LOCO](https://mlsanity.com/paper/2303.05981/) (2023)
- [SHAFF: Fast and consistent SHApley eFfect estimates via random Forests](https://mlsanity.com/paper/2105.11724/) (2021)
- [Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution](https://mlsanity.com/paper/2401.15866/) (2024)
- [Using Shapley Values and Variational Autoencoders to Explain Predictive Models with Dependent Mixed Features](https://mlsanity.com/paper/2111.13507/) (2021)
- [Unifying local and global model explanations by functional decomposition of low dimensional structures](https://mlsanity.com/paper/2208.06151/) (2022)
- [Joint Shapley values: a measure of joint feature importance](https://mlsanity.com/paper/2107.11357/) (2021)
- [A Comparative Study of Methods for Estimating Conditional Shapley Values and When to Use Them](https://mlsanity.com/paper/2305.09536/) (2023)
- [Grouped Feature Importance and Combined Features Effect Plot](https://mlsanity.com/paper/2104.11688/) (2021)
- [Decorrelated Variable Importance](https://mlsanity.com/paper/2111.10853/) (2021)
- [Energy-Based Learning for Cooperative Games, with Applications to Valuation Problems in Machine Learning](https://mlsanity.com/paper/2106.02938/) (2021)
- [Interpretable Machine Learning: Moving From Mythos to Diagnostics](https://mlsanity.com/paper/2103.06254/) (2021)
- [Accurate Shapley Values for explaining tree-based models](https://mlsanity.com/paper/2106.03820/) (2021)
- [Differentially Private Shapley Values for Data Evaluation](https://mlsanity.com/paper/2206.00511/) (2022)
- [Visualizing the Emergence of Intermediate Visual Patterns in DNNs](https://mlsanity.com/paper/2111.03505/) (2021)
- [Ultra-marginal Feature Importance: Learning from Data with Causal Guarantees](https://mlsanity.com/paper/2204.09938/) (2022)
- [Is this model reliable for everyone? Testing for strong calibration](https://mlsanity.com/paper/2307.15247/) (2023)
- [Lazy Estimation of Variable Importance for Large Neural Networks](https://mlsanity.com/paper/2207.09097/) (2022)
- [A hierarchical decomposition for explaining ML performance discrepancies](https://mlsanity.com/paper/2402.14254/) (2024)
- [shapr: Explaining Machine Learning Models with Conditional Shapley Values in R and Python](https://mlsanity.com/paper/2504.01842/) (2025)

## Abstract (arXiv)

The true population-level importance of a variable in a prediction task provides useful knowledge about the underlying data-generating mechanism and can help in deciding which measurements to collect in subsequent experiments. Valid statistical inference on this importance is a key component in understanding the population of interest. We present a computationally efficient procedure for estimating and obtaining valid statistical inference on the Shapley Population Variable Importance Measure (SPVIM). Although the computational complexity of the true SPVIM scales exponentially with the number of variables, we propose an estimator based on randomly sampling only Theta(n) feature subsets given n observations. We prove that our estimator converges at an asymptotically optimal rate. Moreover, by deriving the asymptotic distribution of our estimator, we construct valid confidence intervals and hypothesis tests. Our procedure has good finite-sample performance in simulations, and for an in-hospital mortality prediction task produces similar variable importance estimates when different machine learning algorithms are applied.