mlsanityData to 7 Oct 2026

Prediction Risk and Estimation Risk of the Ridgeless Least Squares Estimator under General Assumptions on Regression Errors

arXiv 2305.12883 ↗First posted 22 May 2023math.ST cs.LG econ.EMcited by 0

Not on the map. Why?

Abstract (arXiv)

In recent years, there has been a significant growth in research focusing on minimum ℓ2 norm (ridgeless) interpolation least squares estimators. However, the majority of these analyses have been limited to an unrealistic regression error structure, assuming independent and identically distributed errors with zero mean and common variance. In this paper, we explore prediction risk as well as estimation risk under more general regression error assumptions, highlighting the benefits of overparameterization in a more realistic setting that allows for clustered or serial dependence. Notably, we establish that the estimation difficulties associated with the variance components of both risks can be summarized through the trace of the variance-covariance matrix of the regression errors. Our findings suggest that the benefits of overparameterization can extend to time series, panel and grouped data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 29 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

16 of 29 references.

References · 16

  1. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  2. 2019Benign Overfitting in Linear Regressioncited by 544
  3. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 508
  4. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 410
  5. 2019Two models of double descent for weak featurescited by 194
  6. 2018To understand deep learning we need to understand kernel learningcited by 186
  7. 2020Benign overfitting in ridge regressioncited by 152
  8. 2015High-Dimensional Asymptotics of Prediction: Ridge Regression and Classificationcited by 146
  9. 2020On the Optimal Weighted ℓ2 Regularization in Overparameterized Linear Regressioncited by 97
  10. 2021Benign Overfitting of Constant-Stepsize SGD for Linear Regressioncited by 71
  11. 2020Asymptotics of Ridge (less) Regression under General Source Conditioncited by 61
  12. 2018Optimal ridge penalty for real-world high-dimensional data can be zero or negative due to the implicit ridge regularizationcited by 52
4 more
  1. 2020Tight Nonparametric Convergence Rates for Stochastic Gradient Descent under the Noiseless Linear Modelcited by 39
  2. 2020On the robustness of minimum norm interpolators and regularized empirical risk minimizerscited by 18
  3. 2023Sketched Ridgeless Linear Regression: The Role of Downsamplingcited by 7
  4. 2023Double and Single Descent in Causal Inference with an Application to High-Dimensional Synthetic Controlcited by 4

Built on by

Not cited yet.