# Knoop: Practical Enhancement of Knockoff with Over-Parameterization for Variable Selection

arXiv: https://arxiv.org/abs/2501.17889 · HTML: https://mlsanity.com/paper/2501.17889/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 28 Jan 2025.

## Builds on (most cited first, 16 of 16)

- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [Just Interpolate: Kernel "Ridgeless" Regression Can Generalize](https://mlsanity.com/paper/1808.00387/) (2018)
- [Benign overfitting in ridge regression](https://mlsanity.com/paper/2009.14286/) (2020)
- [Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection](https://mlsanity.com/paper/1610.02351/) (2016)
- [Controlling the false discovery rate via knockoffs](https://mlsanity.com/paper/1404.5609/) (2014)
- [Asymptotics of Ridge (less) Regression under General Source Condition](https://mlsanity.com/paper/2006.06386/) (2020)
- [Deep Knockoffs](https://mlsanity.com/paper/1811.06687/) (2018)
- [Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders](https://mlsanity.com/paper/2012.00560/) (2020)
- [Testing Conditional Independence in Supervised Learning Algorithms](https://mlsanity.com/paper/1901.09917/) (2019)
- [A knockoff filter for high-dimensional selective inference](https://mlsanity.com/paper/1602.03574/) (2016)
- [Causality-based Feature Selection: Methods and Evaluations](https://mlsanity.com/paper/1911.07147/) (2019)
- [Metropolized Knockoff Sampling](https://mlsanity.com/paper/1903.00434/) (2019)
- [Improving the Stability of the Knockoff Procedure: Multiple Simultaneous Knockoffs and Entropy Maximization](https://mlsanity.com/paper/1810.11378/) (2018)
- [Aggregation of Multiple Knockoffs](https://mlsanity.com/paper/2002.09269/) (2020)
- [Fast and Scalable Lasso via Stochastic Frank-Wolfe Methods with a Convergence Guarantee](https://mlsanity.com/paper/1510.07169/) (2015)
- [Multilayer Knockoff Filter: Controlled variable selection at multiple resolutions](https://mlsanity.com/paper/1706.09375/) (2017)

## Built on by (most cited first, 1 of 1)

- [Knockoff-Guided Compressive Sensing: A Statistical Machine Learning Framework for Support-Assured Signal Recovery](https://mlsanity.com/paper/2505.24727/) (2025)

## Abstract (arXiv)

Variable selection plays a crucial role in enhancing modeling effectiveness across diverse fields, addressing the challenges posed by high-dimensional datasets of correlated variables. This work introduces a novel approach namely Knockoff with over-parameterization (Knoop) to enhance Knockoff filters for variable selection. Specifically, Knoop first generates multiple knockoff variables for each original variable and integrates them with the original variables into an over-parameterized Ridgeless regression model. For each original variable, Knoop evaluates the coefficient distribution of its knockoffs and compares these with the original coefficients to conduct an anomaly-based significance test, ensuring robust variable selection. Extensive experiments demonstrate superior performance compared to existing methods in both simulation and real-world datasets. Knoop achieves a notably higher Area under the Curve (AUC) of the Receiver Operating Characteristic (ROC) Curve for effectively identifying relevant variables against the ground truth by controlled simulations, while showcasing enhanced predictive accuracy across diverse regression and classification tasks. The analytical results further backup our observations.