# Deep Direct Likelihood Knockoffs

arXiv: https://arxiv.org/abs/2007.15835 · HTML: https://mlsanity.com/paper/2007.15835/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2020.

## Built on by (most cited first, 13 of 13)

- [Composite Feature Selection using Deep Ensembles](https://mlsanity.com/paper/2211.00631/) (2022)
- [Survival Mixture Density Networks](https://mlsanity.com/paper/2208.10759/) (2022)
- [Conditional Feature Importance for Mixed Data](https://mlsanity.com/paper/2210.03047/) (2022)
- [Normalizing Flows for Knockoff-free Controlled Feature Selection](https://mlsanity.com/paper/2106.01528/) (2021)
- [Multivariate rank via entropic optimal transport: sample efficiency and generative modeling](https://mlsanity.com/paper/2111.00043/) (2021)
- [DIET: Conditional independence testing with marginal dependence measures of residual information](https://mlsanity.com/paper/2208.08579/) (2022)
- [On the Feasibility of Machine Learning Augmented Magnetic Resonance for Point-of-Care Identification of Disease](https://mlsanity.com/paper/2301.11962/) (2023)
- [Explanations that reveal all through the definition of encoding](https://mlsanity.com/paper/2411.02664/) (2024)
- [Learning to Increase the Power of Conditional Randomization Tests](https://mlsanity.com/paper/2207.01022/) (2022)
- [DeepROCK: Error-controlled interaction detection in deep neural networks](https://mlsanity.com/paper/2309.15319/) (2023)
- [DeepDRK: Deep Dependency Regularized Knockoff for Feature Selection](https://mlsanity.com/paper/2402.17176/) (2024)
- [Empirically Calibrated Conditional Independence Tests](https://mlsanity.com/paper/2602.21036/) (2026)
- [A dynamic risk score for early prediction of cardiogenic shock using machine learning](https://mlsanity.com/paper/2303.12888/) (2023)

## Abstract (arXiv)

Predictive modeling often uses black box machine learning methods, such as deep neural networks, to achieve state-of-the-art performance. In scientific domains, the scientist often wishes to discover which features are actually important for making the predictions. These discoveries may lead to costly follow-up experiments and as such it is important that the error rate on discoveries is not too high. Model-X knockoffs enable important features to be discovered with control of the FDR. However, knockoffs require rich generative models capable of accurately modeling the knockoff features while ensuring they obey the so-called "swap" property. We develop Deep Direct Likelihood Knockoffs (DDLK), which directly minimizes the KL divergence implied by the knockoff swap property. DDLK consists of two stages: it first maximizes the explicit likelihood of the features, then minimizes the KL divergence between the joint distribution of features and knockoffs and any swap between them. To ensure that the generated knockoffs are valid under any possible swap, DDLK uses the Gumbel-Softmax trick to optimize the knockoff generator under the worst-case swap. We find DDLK has higher power than baselines while controlling the false discovery rate on a variety of synthetic and real benchmarks including a task involving a large dataset from one of the epicenters of COVID-19.