# Fast and Powerful Conditional Randomization Testing via Distillation

arXiv: https://arxiv.org/abs/2006.03980 · HTML: https://mlsanity.com/paper/2006.03980/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2020.

## Built on by (most cited first, 19 of 19)

- [Model-X Sequential Testing for Conditional Independence via Testing by Betting](https://mlsanity.com/paper/2210.00354/) (2022)
- [Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning](https://mlsanity.com/paper/2102.12328/) (2021)
- [Feature Selection for Discovering Distributional Treatment Effect Modifiers](https://mlsanity.com/paper/2206.00516/) (2022)
- [Conditional independence testing under misspecified inductive biases](https://mlsanity.com/paper/2307.02520/) (2023)
- [Statistically Valid Variable Importance Assessment through Conditional Permutations](https://mlsanity.com/paper/2309.07593/) (2023)
- [SHAP-XRT: The Shapley Value Meets Conditional Independence Testing](https://mlsanity.com/paper/2207.07038/) (2022)
- [Normalizing Flows for Knockoff-free Controlled Feature Selection](https://mlsanity.com/paper/2106.01528/) (2021)
- [General Frameworks for Conditional Two-Sample Testing](https://mlsanity.com/paper/2410.16636/) (2024)
- [DIET: Conditional independence testing with marginal dependence measures of residual information](https://mlsanity.com/paper/2208.08579/) (2022)
- [Near-optimal multiple testing in Bayesian linear models with finite-sample FDR control](https://mlsanity.com/paper/2211.02778/) (2022)
- [A Conditional Randomization Test for Sparse Logistic Regression in High-Dimension](https://mlsanity.com/paper/2205.14613/) (2022)
- [Learning to Increase the Power of Conditional Randomization Tests](https://mlsanity.com/paper/2207.01022/) (2022)
- [Which Sparse Autoencoder Features Are Real? Model-X Knockoffs for False Discovery Rate Control](https://mlsanity.com/paper/2511.11711/) (2025)
- [Spatially relaxed inference on high-dimensional linear models](https://mlsanity.com/paper/2106.02590/) (2021)
- [Using Machine Learning to Test Causal Hypotheses in Conjoint Analysis](https://mlsanity.com/paper/2201.08343/) (2022)
- [FACT: High-Dimensional Random Forests Inference](https://mlsanity.com/paper/2207.01678/) (2022)
- [A Model-free Closeness-of-influence Test for Features in Supervised Learning](https://mlsanity.com/paper/2306.11855/) (2023)
- [Stability of a Generalized Debiased Lasso with Applications to Resampling-Based Variable Selection](https://mlsanity.com/paper/2405.03063/) (2024)
- [AICO: Feature Significance Tests for Supervised Learning](https://mlsanity.com/paper/2506.23396/) (2025)

## Abstract (arXiv)

We consider the problem of conditional independence testing: given a response Y and covariates (X,Z), we test the null hypothesis that Y is independent of X given Z. The conditional randomization test (CRT) was recently proposed as a way to use distributional information about X|Z to exactly (non-asymptotically) control Type-I error using any test statistic in any dimensionality without assuming anything about Y|(X,Z). This flexibility in principle allows one to derive powerful test statistics from complex prediction algorithms while maintaining statistical validity. Yet the direct use of such advanced test statistics in the CRT is prohibitively computationally expensive, especially with multiple testing, due to the CRT's requirement to recompute the test statistic many times on resampled data. We propose the distilled CRT, a novel approach to using state-of-the-art machine learning algorithms in the CRT while drastically reducing the number of times those algorithms need to be run, thereby taking advantage of their power and the CRT's statistical guarantees without suffering the usual computational expense. In addition to distillation, we propose a number of other tricks like screening and recycling computations to further speed up the CRT without sacrificing its high power and exact validity. Indeed, we show in simulations that all our proposals combined lead to a test that has similar power to the most powerful existing CRT implementations but requires orders of magnitude less computation, making it a practical tool even for large data sets. We demonstrate these benefits on a breast cancer dataset by identifying biomarkers related to cancer stage.