# A general framework for inference on algorithm-agnostic variable importance

arXiv: https://arxiv.org/abs/2004.03683 · HTML: https://mlsanity.com/paper/2004.03683/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2020.

## Built on by (most cited first, 30 of 45)

- [The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importance](https://mlsanity.com/paper/2309.13775/) (2023)
- [One-Step Estimation of Differentiable Hilbert-Valued Parameters](https://mlsanity.com/paper/2303.16711/) (2023)
- [A Guide to Feature Importance Methods for Scientific Inference](https://mlsanity.com/paper/2404.12862/) (2024)
- [MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA](https://mlsanity.com/paper/2102.13347/) (2021)
- [Interpretable Machine Learning for Discovery: Statistical Challenges & Opportunities](https://mlsanity.com/paper/2308.01475/) (2023)
- [Statistically Valid Variable Importance Assessment through Conditional Permutations](https://mlsanity.com/paper/2309.07593/) (2023)
- [Variable importance for causal forests: breaking down the heterogeneity of treatment effects](https://mlsanity.com/paper/2308.03369/) (2023)
- [Tree Ensemble Explainability through the Hoeffding Functional Decomposition and TreeHFD Algorithm](https://mlsanity.com/paper/2510.24815/) (2025)
- [Lazy Estimation of Variable Importance for Large Neural Networks](https://mlsanity.com/paper/2207.09097/) (2022)
- [ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations](https://mlsanity.com/paper/2601.23068/) (2026)
- [Variable Importance in High-Dimensional Settings Requires Grouping](https://mlsanity.com/paper/2312.10858/) (2023)
- [General Frameworks for Conditional Two-Sample Testing](https://mlsanity.com/paper/2410.16636/) (2024)
- [Measuring Variable Importance in Heterogeneous Treatment Effects with Confidence](https://mlsanity.com/paper/2408.13002/) (2024)
- [Deep Nonparametric Conditional Independence Tests for Images](https://mlsanity.com/paper/2411.06140/) (2024)
- [Reliable and scalable variable importance estimation via warm-start and early stopping](https://mlsanity.com/paper/2412.01120/) (2024)
- [iLOCO: Distribution-Free Inference for Feature Interactions](https://mlsanity.com/paper/2502.06661/) (2025)
- [Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect](https://mlsanity.com/paper/2510.12734/) (2025)
- [MMD-based Variable Importance for Distributional Random Forest](https://mlsanity.com/paper/2310.12115/) (2023)
- [Factor Importance Ranking and Selection using Total Indices](https://mlsanity.com/paper/2401.00800/) (2024)
- [Model-independent variable selection via the rule-based variable priority](https://mlsanity.com/paper/2409.09003/) (2024)
- [Inference on Variable Importance for Treatment Effect Heterogeneity: Shapley Values and Beyond](https://mlsanity.com/paper/2510.18843/) (2025)
- [Flexible variable selection in the presence of missing data](https://mlsanity.com/paper/2202.12989/) (2022)
- [Measuring the Driving Forces of Predictive Performance: Application to Credit Scoring](https://mlsanity.com/paper/2212.05866/) (2022)
- [Double Debiased Machine Learning for Mediation Analysis with Continuous Treatments](https://mlsanity.com/paper/2503.06156/) (2025)
- [Statistical Inference and Learning for Shapley Additive Explanations (SHAP)](https://mlsanity.com/paper/2602.10532/) (2026)
- [Aggregate Models, Not Explanations: Improving Feature Importance Estimation](https://mlsanity.com/paper/2602.11760/) (2026)
- [xplainfi: Feature Importance and Statistical Inference for Machine Learning in R](https://mlsanity.com/paper/2603.15306/) (2026)
- [Regional Explanations: Bridging Local and Global Variable Importance](https://mlsanity.com/paper/2604.11223/) (2026)
- [MinShap: A Shapley-Based Framework for Feature Redundancy](https://mlsanity.com/paper/2604.15107/) (2026)
- [The Debiased Score Test: Hunt-and-test for Semiparametric Hypotheses](https://mlsanity.com/paper/2607.28861/) (2026)

## Abstract (arXiv)

In many applications, it is of interest to assess the relative contribution of features (or subsets of features) toward the goal of predicting a response -- in other words, to gauge the variable importance of features. Most recent work on variable importance assessment has focused on describing the importance of features within the confines of a given prediction algorithm. However, such assessment does not necessarily characterize the prediction potential of features, and may provide a misleading reflection of the intrinsic value of these features. To address this limitation, we propose a general framework for nonparametric inference on interpretable algorithm-agnostic variable importance. We define variable importance as a population-level contrast between the oracle predictiveness of all available features versus all features except those under consideration. We propose a nonparametric efficient estimation procedure that allows the construction of valid confidence intervals, even when machine learning techniques are used. We also outline a valid strategy for testing the null importance hypothesis. Through simulations, we show that our proposal has good operating characteristics, and we illustrate its use with data from a study of an antibody against HIV-1 infection.