# Testing Conditional Independence in Supervised Learning Algorithms

arXiv: https://arxiv.org/abs/1901.09917 · HTML: https://mlsanity.com/paper/1901.09917/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jan 2019.

## Built on by (most cited first, 27 of 27)

- [Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Process](https://mlsanity.com/paper/2109.01433/) (2021)
- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [Grouped Feature Importance and Combined Features Effect Plot](https://mlsanity.com/paper/2104.11688/) (2021)
- [A Guide to Feature Importance Methods for Scientific Inference](https://mlsanity.com/paper/2404.12862/) (2024)
- [Conditional Feature Importance for Mixed Data](https://mlsanity.com/paper/2210.03047/) (2022)
- [Conditional independence testing under misspecified inductive biases](https://mlsanity.com/paper/2307.02520/) (2023)
- [Statistically Valid Variable Importance Assessment through Conditional Permutations](https://mlsanity.com/paper/2309.07593/) (2023)
- [Counterfactual Generation with Knockoffs](https://mlsanity.com/paper/2102.00951/) (2021)
- [A Simple Unified Approach to Testing High-Dimensional Conditional Independences for Categorical and Ordinal Data](https://mlsanity.com/paper/2206.04356/) (2022)
- [Variable Importance in High-Dimensional Settings Requires Grouping](https://mlsanity.com/paper/2312.10858/) (2023)
- [Interpretable Machine Learning for Survival Analysis](https://mlsanity.com/paper/2403.10250/) (2024)
- [Deep Nonparametric Conditional Independence Tests for Images](https://mlsanity.com/paper/2411.06140/) (2024)
- [Reliable and scalable variable importance estimation via warm-start and early stopping](https://mlsanity.com/paper/2412.01120/) (2024)
- [iLOCO: Distribution-Free Inference for Feature Interactions](https://mlsanity.com/paper/2502.06661/) (2025)
- [PPFS: Predictive Permutation Feature Selection](https://mlsanity.com/paper/2110.10713/) (2021)
- [Explaining and visualizing black-box models through counterfactual paths](https://mlsanity.com/paper/2307.07764/) (2023)
- [Opening the random forest black box by the analysis of the mutual impact of features](https://mlsanity.com/paper/2304.02490/) (2023)
- [Knoop: Practical Enhancement of Knockoff with Over-Parameterization for Variable Selection](https://mlsanity.com/paper/2501.17889/) (2025)
- [What's Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Models](https://mlsanity.com/paper/2504.20687/) (2025)
- [Decorrelated feature importance from local sample weighting](https://mlsanity.com/paper/2508.06337/) (2025)
- [xplainfi: Feature Importance and Statistical Inference for Machine Learning in R](https://mlsanity.com/paper/2603.15306/) (2026)
- [Challenges in Variable Importance Ranking Under Correlation](https://mlsanity.com/paper/2402.03447/) (2024)
- [Inference of Causal Networks using a Topological Threshold](https://mlsanity.com/paper/2404.14460/) (2024)
- [AICO: Feature Significance Tests for Supervised Learning](https://mlsanity.com/paper/2506.23396/) (2025)
- [Provable Recovery of Locally Important Signed Features and Interactions from Random Forest](https://mlsanity.com/paper/2512.11081/) (2025)
- [Adaptive-CaRe: Adaptive Causal Regularization for Robust Outcome Prediction](https://mlsanity.com/paper/2602.06611/) (2026)
- [PAIR-CI: Calibrated Conditional Independence Testing for Causal Discovery with Incomplete Data](https://mlsanity.com/paper/2605.04838/) (2026)

## Abstract (arXiv)

We propose the conditional predictive impact (CPI), a consistent and unbiased estimator of the association between one or several features and a given outcome, conditional on a reduced feature set. Building on the knockoff framework of Cand`es et al. (2018), we develop a novel testing procedure that works in conjunction with any valid knockoff sampler, supervised learning algorithm, and loss function. The CPI can be efficiently computed for high-dimensional data without any sparsity constraints. We demonstrate convergence criteria for the CPI and develop statistical inference procedures for evaluating its magnitude, significance, and precision. These tests aid in feature and model selection, extending traditional frequentist and Bayesian techniques to general supervised learning tasks. The CPI may also be applied in causal discovery to identify underlying multivariate graph structures. We test our method using various algorithms, including linear regression, neural networks, random forests, and support vector machines. Empirical results show that the CPI compares favorably to alternative variable importance measures and other nonparametric tests of conditional independence on a diverse array of real and simulated datasets. Simulations confirm that our inference procedures successfully control Type I error and achieve nominal coverage probability. Our method has been implemented in an R package, cpi, which can be downloaded from https://github.com/dswatson/cpi.