# Asymptotic Optimism of Random-Design Linear and Kernel Regression Models

arXiv: https://arxiv.org/abs/2502.12999 · HTML: https://mlsanity.com/paper/2502.12999/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2025.

## Built on by (most cited first, 3 of 3)

- [GGMPs: Generalized Gaussian Mixture Processes](https://mlsanity.com/paper/2603.10442/) (2026)
- [A Special Case of Quadratic Extrapolation Under the Neural Tangent Kernel](https://mlsanity.com/paper/2512.15749/) (2025)
- [Asymptotic Optimism for Tensor Regression Models with Applications to Neural Network Compression](https://mlsanity.com/paper/2603.26048/) (2026)

## Abstract (arXiv)

We derived the closed-form asymptotic optimism of linear regression models under random designs, and generalizes it to kernel ridge regression. Using scaled asymptotic optimism as a generic predictive model complexity measure, we studied the fundamental different behaviors of linear regression model, tangent kernel (NTK) regression model and three-layer fully connected neural networks (NN). Our contribution is two-fold: we provided theoretical ground for using scaled optimism as a model predictive complexity measure; and we show empirically that NN with ReLUs behaves differently from kernel models under this measure. With resampling techniques, we can also compute the optimism for regression models with real data.