# Powerful Knockoffs via Minimizing Reconstructability

arXiv: https://arxiv.org/abs/2011.14625 · HTML: https://mlsanity.com/paper/2011.14625/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2020.

## Built on by (most cited first, 6 of 6)

- [Near-optimal multiple testing in Bayesian linear models with finite-sample FDR control](https://mlsanity.com/paper/2211.02778/) (2022)
- [NeurFlow: Interpreting Neural Networks through Neuron Groups and Functional Interactions](https://mlsanity.com/paper/2502.16105/) (2025)
- [Data-driven sparse identification of governing PDEs via knockoff filters and multi-criteria trade-offs](https://mlsanity.com/paper/2605.26631/) (2026)
- [DeepDRK: Deep Dependency Regularized Knockoff for Feature Selection](https://mlsanity.com/paper/2402.17176/) (2024)
- [Variable Selection with the Knockoffs: Composite Null Hypotheses](https://mlsanity.com/paper/2203.02849/) (2022)
- [Virtual Dummies: Enabling Scalable FDR-Controlled Variable Selection via Sequential Sampling of Null Features](https://mlsanity.com/paper/2604.07464/) (2026)

## Abstract (arXiv)

Model-X knockoffs allows analysts to perform feature selection using almost any machine learning algorithm while still provably controlling the expected proportion of false discoveries. To apply model-X knockoffs, one must construct synthetic variables, called knockoffs, which effectively act as controls during feature selection. The gold standard for constructing knockoffs has been to minimize the mean absolute correlation (MAC) between features and their knockoffs, but, surprisingly, we prove this procedure can be powerless in extremely easy settings, including Gaussian linear models with correlated exchangeable features. The key problem is that minimizing the MAC creates strong joint dependencies between the features and knockoffs, which allow machine learning algorithms to partially or fully reconstruct the effect of the features on the response using the knockoffs. To improve the power of knockoffs, we propose generating knockoffs which minimize the reconstructability (MRC) of the features, and we demonstrate our proposal for Gaussian features by showing it is computationally efficient, robust, and powerful. We also prove that certain MRC knockoffs minimize a natural definition of estimation error in Gaussian linear models. Furthermore, in an extensive set of simulations, we find many settings with correlated features in which MRC knockoffs dramatically outperform MAC-minimizing knockoffs and no settings in which MAC-minimizing knockoffs outperform MRC knockoffs by more than a very slight margin. We implement our methods and a host of others from the knockoffs literature in a new open source python package knockpy.