# Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection

arXiv: https://arxiv.org/abs/1610.02351 · HTML: https://mlsanity.com/paper/1610.02351/

## Key facts

- Cited by 121 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2016.

## Built on by (most cited first, 30 of 121)

- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experiments](https://mlsanity.com/paper/2206.10013/) (2022)
- [Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Value](https://mlsanity.com/paper/2304.07718/) (2023)
- [A Data-Based Perspective on Transfer Learning](https://mlsanity.com/paper/2207.05739/) (2022)
- [Practical Kernel Tests of Conditional Independence](https://mlsanity.com/paper/2402.13196/) (2024)
- [Knockoff-Guided Feature Selection via A Single Pre-trained Reinforced Agent](https://mlsanity.com/paper/2403.04015/) (2024)
- [Model-X Sequential Testing for Conditional Independence via Testing by Betting](https://mlsanity.com/paper/2210.00354/) (2022)
- [An Asymptotic Test for Conditional Independence using Analytic Kernel Embeddings](https://mlsanity.com/paper/2110.14868/) (2021)
- [Composite Feature Selection using Deep Ensembles](https://mlsanity.com/paper/2211.00631/) (2022)
- [The Terminating-Random Experiments Selector: Fast High-Dimensional Variable Selection with False Discovery Rate Control](https://mlsanity.com/paper/2110.06048/) (2021)
- [Statistical quantification of confounding bias in predictive modelling](https://mlsanity.com/paper/2111.00814/) (2021)
- [Feature Selection for Discovering Distributional Treatment Effect Modifiers](https://mlsanity.com/paper/2206.00516/) (2022)
- [Conditional Feature Importance for Mixed Data](https://mlsanity.com/paper/2210.03047/) (2022)
- [Statistically Valid Variable Importance Assessment through Conditional Permutations](https://mlsanity.com/paper/2309.07593/) (2023)
- [Deep anytime-valid hypothesis testing](https://mlsanity.com/paper/2310.19384/) (2023)
- [Doubly Robust Conditional Independence Testing with Generative Neural Networks](https://mlsanity.com/paper/2407.17694/) (2024)
- [On the Hardness of Conditional Independence Testing In Practice](https://mlsanity.com/paper/2512.14000/) (2025)
- [Causal discovery under a confounder blanket](https://mlsanity.com/paper/2205.05715/) (2022)
- [SHAP-XRT: The Shapley Value Meets Conditional Independence Testing](https://mlsanity.com/paper/2207.07038/) (2022)
- [Nearest-Neighbor Sampling Based Conditional Independence Testing](https://mlsanity.com/paper/2304.04183/) (2023)
- [FDR-Controlled Portfolio Optimization for Sparse Financial Index Tracking](https://mlsanity.com/paper/2401.15139/) (2024)
- [Efficient Ensemble Conditional Independence Test Framework for Causal Discovery](https://mlsanity.com/paper/2509.21021/) (2025)
- [Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guarantees](https://mlsanity.com/paper/2204.07293/) (2022)
- [Causal Discovery using Model Invariance through Knockoff Interventions](https://mlsanity.com/paper/2207.04055/) (2022)
- [Statistically Significant Concept-based Explanation of Image Classifiers via Model Knockoffs](https://mlsanity.com/paper/2305.18362/) (2023)
- [Variable Importance in High-Dimensional Settings Requires Grouping](https://mlsanity.com/paper/2312.10858/) (2023)
- [Sparse PCA with False Discovery Rate Controlled Variable Selection](https://mlsanity.com/paper/2401.08375/) (2024)
- [FairICP: Encouraging Equalized Odds via Inverse Conditional Permutation](https://mlsanity.com/paper/2404.05678/) (2024)
- [General Frameworks for Conditional Two-Sample Testing](https://mlsanity.com/paper/2410.16636/) (2024)
- [Taming Nonconvexity in Kernel Feature Selection -- Favorable Properties of the Laplace Kernel](https://mlsanity.com/paper/2106.09387/) (2021)

## Abstract (arXiv)

Many contemporary large-scale applications involve building interpretable models linking a large set of potential covariates to a response in a nonlinear fashion, such as when the response is binary. Although this modeling problem has been extensively studied, it remains unclear how to effectively control the fraction of false discoveries even in high-dimensional logistic regression, not to mention general high-dimensional nonlinear models. To address such a practical problem, we propose a new framework of model-X knockoffs, which reads from a different perspective the knockoff procedure (Barber and Cand`es, 2015) originally designed for controlling the false discovery rate in linear models. Whereas the knockoffs procedure is constrained to homoscedastic linear models with n≥ p, the key innovation here is that model-X knockoffs provide valid inference from finite samples in settings in which the conditional distribution of the response is arbitrary and completely unknown. Furthermore, this holds no matter the number of covariates. Correct inference in such a broad setting is achieved by constructing knockoff variables probabilistically instead of geometrically. To do this, our approach requires the covariates be random (independent and identically distributed rows) with a distribution that is known, although we provide preliminary experimental evidence that our procedure is robust to unknown/estimated distributions. To our knowledge, no other procedure solves the controlled variable selection problem in such generality, but in the restricted settings where competitors exist, we demonstrate the superior power of knockoffs through simulations. Finally, we apply our procedure to data from a case-control study of Crohn's disease in the United Kingdom, making twice as many discoveries as the original analysis of the same data.