Optimal ridge regularization revisited
Not on the map. Why?
Abstract (arXiv)
We consider L2-regularized linear (ridge) regression over a finite data sample X with bounded covariance and linear prediction targets y with additive isotropic noise of finite variance. We present an iterative procedure to compute the optimal regularization strength numerically from the generative parameters in the fixed-X setting and prove its convergence at limited noise levels. Our experimental evaluation over synthetic data shows that the proposed procedure combined with sample-based parameter estimates attains near-optimal random-X generalization across a wide range of sample sizes, aspect ratios, and noise levels, at an added computational cost equivalent to one preliminary ridge regression in the underparameterized regime and two in the overparameterized case.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 8 of its 26 references that are in the map. First posted on arXiv 27 May 2026.
Builds on
8 of 26 references.
References · 8
- 2020Benign overfitting in ridge regressioncited by 152
- 2015High-Dimensional Asymptotics of Prediction: Ridge Regression and Classificationcited by 146
- 2020On the Optimal Weighted ℓ2 Regularization in Overparameterized Linear Regressioncited by 97
- 2018Optimal ridge penalty for real-world high-dimensional data can be zero or negative due to the implicit ridge regularizationcited by 52
- 2020On the training dynamics of deep networks with L2 regularizationcited by 38
- 2020Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Networkcited by 38
- 2024Optimal Ridge Regularization for Out-of-Distribution Predictioncited by 18
- 2017From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimationcited by 12
Built on by
Not cited yet.