# Asymptotic errors for convex penalized linear regression beyond Gaussian matrices

arXiv: https://arxiv.org/abs/2002.04372 · HTML: https://mlsanity.com/paper/2002.04372/

## Key facts

- Cited by 6 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 6 of 6)

- [Implicit Bias of Linear RNNs](https://mlsanity.com/paper/2101.07833/) (2021)
- [Performance of Bayesian linear regression in a model with mismatch](https://mlsanity.com/paper/2107.06936/) (2021)
- [Double-descent curves in neural networks: a new perspective using Gaussian processes](https://mlsanity.com/paper/2102.07238/) (2021)
- [Observable adjustments in single-index models for regularized M-estimators](https://mlsanity.com/paper/2204.06990/) (2022)
- [Ising Model Selection Using ℓ1-Regularized Linear Regression: A Statistical Mechanics Analysis](https://mlsanity.com/paper/2102.03988/) (2021)
- [Impact of Bottleneck Layers and Skip Connections on the Generalization of Linear Denoising Autoencoders](https://mlsanity.com/paper/2505.24668/) (2025)

## Abstract (arXiv)

We consider the problem of learning a coefficient vector x0 in RN from noisy linear observations y=Fx0+w in RM in the high dimensional limit M,N to infinity with α=M/N fixed. We provide a rigorous derivation of an explicit formula -- first conjectured using heuristic methods from statistical physics -- for the asymptotic mean squared error obtained by penalized convex regression estimators such as the LASSO or the elastic net, for a class of very generic random matrices corresponding to rotationally invariant data matrices with arbitrary spectrum. The proof is based on a convergence analysis of an oracle version of vector approximate message-passing (oracle-VAMP) and on the properties of its state evolution equations. Our method leverages on and highlights the link between vector approximate message-passing, Douglas-Rachford splitting and proximal descent algorithms, extending previous results obtained with i.i.d. matrices for a large class of problems. We illustrate our results on some concrete examples and show that even though they are asymptotic, our predictions agree remarkably well with numerics even for very moderate sizes.