# σ-Ridge: group regularized ridge regression via empirical Bayes noise level cross-validation

arXiv: https://arxiv.org/abs/2010.15817 · HTML: https://mlsanity.com/paper/2010.15817/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2020.

## Built on by (most cited first, 2 of 2)

- [ecpc: An R-package for generic co-data models for high-dimensional prediction](https://mlsanity.com/paper/2205.07640/) (2022)
- [Guiding adaptive shrinkage by co-data to improve regression-based prediction and feature selection](https://mlsanity.com/paper/2405.04917/) (2024)

## Abstract (arXiv)

Features in predictive models are not exchangeable, yet common supervised models treat them as such. Here we study ridge regression when the analyst can partition the features into K groups based on external side-information. For example, in high-throughput biology, features may represent gene expression, protein abundance or clinical data and so each feature group represents a distinct modality. The analyst's goal is to choose optimal regularization parameters λ = (λ1, dotsc, λK) -- one for each group. In this work, we study the impact of λ on the predictive risk of group-regularized ridge regression by deriving limiting risk formulae under a high-dimensional random effects model with pasymp n as n → ∞. Furthermore, we propose a data-driven method for choosing λ that attains the optimal asymptotic risk: The key idea is to interpret the residual noise variance σ2, as a regularization parameter to be chosen through cross-validation. An empirical Bayes construction maps the one-dimensional parameter σ to the K-dimensional vector of regularization parameters, i.e., σ mapsto widehatλ(σ). Beyond its theoretical optimality, the proposed method is practical and runs as fast as cross-validated ridge regression without feature groups (K=1).