# Ridge Regression Revisited: Debiasing, Thresholding and Bootstrap

arXiv: https://arxiv.org/abs/2009.08071 · HTML: https://mlsanity.com/paper/2009.08071/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2020.

## Built on by (most cited first, 2 of 2)

- [Benign overfitting beyond prediction: The ordinary least squares interpolator](https://mlsanity.com/paper/2309.15769/) (2023)
- [Optimal Bias-Correction and Valid Inference in High-Dimensional Ridge Regression: A Closed-Form Solution](https://mlsanity.com/paper/2405.00424/) (2024)

## Abstract (arXiv)

The success of the Lasso in the era of high-dimensional data can be attributed to its conducting an implicit model selection, i.e., zeroing out regression coefficients that are not significant. By contrast, classical ridge regression can not reveal a potential sparsity of parameters, and may also introduce a large bias under the high-dimensional setting. Nevertheless, recent work on the Lasso involves debiasing and thresholding, the latter in order to further enhance the model selection. As a consequence, ridge regression may be worth another look since -- after debiasing and thresholding -- it may offer some advantages over the Lasso, e.g., it can be easily computed using a closed-form expression. % and it has similar performance to threshold Lasso. In this paper, we define a debiased and thresholded ridge regression method, and prove a consistency result and a Gaussian approximation theorem. We further introduce a wild bootstrap algorithm to construct confidence regions and perform hypothesis testing for a linear combination of parameters. In addition to estimation, we consider the problem of prediction, and present a novel, hybrid bootstrap algorithm tailored for prediction intervals. Extensive numerical simulations further show that the debiased and thresholded ridge regression has favorable finite sample performance and may be preferable in some settings.