# DeepDRK: Deep Dependency Regularized Knockoff for Feature Selection

arXiv: https://arxiv.org/abs/2402.17176 · HTML: https://mlsanity.com/paper/2402.17176/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 70 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Generalized Sliced Wasserstein Distances](https://mlsanity.com/paper/1902.00434/) (2019)
- [Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection](https://mlsanity.com/paper/1610.02351/) (2016)
- [Max-Sliced Wasserstein Distance and its use for GANs](https://mlsanity.com/paper/1904.05877/) (2019)
- [Generative Modeling using the Sliced Wasserstein Distance](https://mlsanity.com/paper/1803.11188/) (2018)
- [Controlling the false discovery rate via knockoffs](https://mlsanity.com/paper/1404.5609/) (2014)
- [Asymptotic Guarantees for Learning Generative Models with the Sliced-Wasserstein Distance](https://mlsanity.com/paper/1906.04516/) (2019)
- [DeepPINK: reproducible feature selection in deep neural networks](https://mlsanity.com/paper/1809.01185/) (2018)
- [Deep Knockoffs](https://mlsanity.com/paper/1811.06687/) (2018)
- [Hierarchical Sliced Wasserstein Distance](https://mlsanity.com/paper/2209.13570/) (2022)
- [Gene Hunting with Knockoffs for Hidden Markov Models](https://mlsanity.com/paper/1706.04677/) (2017)
- [Deep Direct Likelihood Knockoffs](https://mlsanity.com/paper/2007.15835/) (2020)
- [Knockoffs for the mass: new feature importance statistics with false discovery guarantees](https://mlsanity.com/paper/1807.06214/) (2018)
- [Auto-Encoding Knockoff Generator for FDR Controlled Variable Selection](https://mlsanity.com/paper/1809.10765/) (2018)
- [Powerful Knockoffs via Minimizing Reconstructability](https://mlsanity.com/paper/2011.14625/) (2020)
- [Normalizing Flows for Knockoff-free Controlled Feature Selection](https://mlsanity.com/paper/2106.01528/) (2021)
- [Multivariate rank via entropic optimal transport: sample efficiency and generative modeling](https://mlsanity.com/paper/2111.00043/) (2021)

## Built on by (most cited first, 1 of 1)

- [Model-Agnostic FDR Control via Group Gaussian Mirror and Permutation SHAP](https://mlsanity.com/paper/2608.00989/) (2026)

## Abstract (arXiv)

Model-X knockoff has garnered significant attention among various feature selection methods due to its guarantees for controlling the false discovery rate (FDR). Since its introduction in parametric design, knockoff techniques have evolved to handle arbitrary data distributions using deep learning-based generative models. However, we have observed limitations in the current implementations of the deep Model-X knockoff framework. Notably, the "swap property" that knockoffs require often faces challenges at the sample level, resulting in diminished selection power. To address these issues, we develop "Deep Dependency Regularized Knockoff (DeepDRK)," a distribution-free deep learning method that effectively balances FDR and power. In DeepDRK, we introduce a novel formulation of the knockoff model as a learning problem under multi-source adversarial attacks. By employing an innovative perturbation technique, we achieve lower FDR and higher power. Our model outperforms existing benchmarks across synthetic, semi-synthetic, and real-world datasets, particularly when sample sizes are small and data distributions are non-Gaussian.