# Floodgate: inference for model-free variable importance

arXiv: https://arxiv.org/abs/2007.01283 · HTML: https://mlsanity.com/paper/2007.01283/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 16 of 16)

- [Cross-validation: what does it estimate and how well does it do it?](https://mlsanity.com/paper/2104.00673/) (2021)
- [The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importance](https://mlsanity.com/paper/2309.13775/) (2023)
- [Optimal ablation for interpretability](https://mlsanity.com/paper/2409.09951/) (2024)
- [Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Process](https://mlsanity.com/paper/2109.01433/) (2021)
- [Feature Importance: A Closer Look at Shapley Values and LOCO](https://mlsanity.com/paper/2303.05981/) (2023)
- [Decorrelated Variable Importance](https://mlsanity.com/paper/2111.10853/) (2021)
- [Rethinking Explainable Machine Learning as Applied Statistics](https://mlsanity.com/paper/2402.02870/) (2024)
- [Conditional independence testing under misspecified inductive biases](https://mlsanity.com/paper/2307.02520/) (2023)
- [Interpretable Machine Learning for Discovery: Statistical Challenges & Opportunities](https://mlsanity.com/paper/2308.01475/) (2023)
- [Scalable and Flexible Causal Discovery with an Efficient Test for Adjacency](https://mlsanity.com/paper/2406.09177/) (2024)
- [Lazy Estimation of Variable Importance for Large Neural Networks](https://mlsanity.com/paper/2207.09097/) (2022)
- [Model free variable importance for high dimensional data](https://mlsanity.com/paper/2211.08414/) (2022)
- [Reliable and scalable variable importance estimation via warm-start and early stopping](https://mlsanity.com/paper/2412.01120/) (2024)
- [iLOCO: Distribution-Free Inference for Feature Interactions](https://mlsanity.com/paper/2502.06661/) (2025)
- [Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect](https://mlsanity.com/paper/2510.12734/) (2025)
- [LOCO-AdaMP: Built-in LOCO Inference for Adaptive Minipatch Ensembles with Enhanced Prediction](https://mlsanity.com/paper/2609.36396/) (2026)

## Abstract (arXiv)

Many modern applications seek to understand the relationship between an outcome variable Y and a covariate X in the presence of a (possibly high-dimensional) confounding variable Z. Although much attention has been paid to testing whether Y depends on X given Z, in this paper we seek to go beyond testing by inferring the strength of that dependence. We first define our estimand, the minimum mean squared error (mMSE) gap, which quantifies the conditional relationship between Y and X in a way that is deterministic, model-free, interpretable, and sensitive to nonlinearities and interactions. We then propose a new inferential approach called floodgate that can leverage any working regression function chosen by the user (allowing, e.g., it to be fitted by a state-of-the-art machine learning algorithm or be derived from qualitative domain knowledge) to construct asymptotic confidence bounds, and we apply it to the mMSE gap. accWe additionally show that floodgate's accuracy (distance from confidence bound to estimand) is adaptive to the error of the working regression function. We then show we can apply the same floodgate principle to a different measure of variable importance when Y is binary. Finally, we demonstrate floodgate's performance in a series of simulations and apply it to data from the UK Biobank to infer the strengths of dependence of platelet count on various groups of genetic mutations.