# Optimal ridge regularization revisited

arXiv: https://arxiv.org/abs/2605.28679 · HTML: https://mlsanity.com/paper/2605.28679/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 26 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 8 of 8)

- [Benign overfitting in ridge regression](https://mlsanity.com/paper/2009.14286/) (2020)
- [High-Dimensional Asymptotics of Prediction: Ridge Regression and Classification](https://mlsanity.com/paper/1507.03003/) (2015)
- [On the Optimal Weighted ℓ2 Regularization in Overparameterized Linear Regression](https://mlsanity.com/paper/2006.05800/) (2020)
- [Optimal ridge penalty for real-world high-dimensional data can be zero or negative due to the implicit ridge regularization](https://mlsanity.com/paper/1805.10939/) (2018)
- [On the training dynamics of deep networks with L2 regularization](https://mlsanity.com/paper/2006.08643/) (2020)
- [Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Network](https://mlsanity.com/paper/2007.02486/) (2020)
- [Optimal Ridge Regularization for Out-of-Distribution Prediction](https://mlsanity.com/paper/2404.01233/) (2024)
- [From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation](https://mlsanity.com/paper/1704.08160/) (2017)

## Abstract (arXiv)

We consider L2-regularized linear (ridge) regression over a finite data sample X with bounded covariance and linear prediction targets y with additive isotropic noise of finite variance. We present an iterative procedure to compute the optimal regularization strength numerically from the generative parameters in the fixed-X setting and prove its convergence at limited noise levels. Our experimental evaluation over synthetic data shows that the proposed procedure combined with sample-based parameter estimates attains near-optimal random-X generalization across a wide range of sample sizes, aspect ratios, and noise levels, at an added computational cost equivalent to one preliminary ridge regression in the underparameterized regime and two in the overparameterized case.