mlsanityData to 7 Oct 2026

An elementary analysis of ridge regression with random design

arXiv 2203.08564 ↗First posted 16 Mar 2022math.ST cs.LG stat.MLcited by 15

Not on the map. Why?

Abstract (arXiv)

In this note, we provide an elementary analysis of the prediction error of ridge regression with random design. The proof is short and self-contained. In particular, it bypasses the use of Rudelson's deviation inequality for covariance matrices, through a combination of exchangeability arguments, matrix perturbation and operator convexity.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 8 of its 43 references that are in the map. First posted on arXiv 16 Mar 2022.

Builds on

8 of 43 references.

References · 8

  1. 2019Benign Overfitting in Linear Regressioncited by 544
  2. 2020Benign overfitting in ridge regressioncited by 152
  3. 2013Non-strongly-convex smooth stochastic approximation with convergence rate O(1/n)cited by 110
  4. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  5. 2019Exact minimax risk for linear least squares, and the lower tail of sample covariance matricescited by 34
  6. 2019An improper estimator with optimal excess risk in misspecified density estimation and logistic regressioncited by 24
  7. 2020Suboptimality of Constrained Least Squares and Improvements via Non-Linear Predictorscited by 9
  8. 2016Average Stability is Invariant to Data Preconditioning. Implications to Exp-concave Empirical Risk Minimizationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 2; 2025: 5; 2026: 4.

15 papers.

Sort:

  1. 2023High-dimensional analysis of double descent for linear regression with random projectionscited by 47
  2. 2023The SSL Interplay: Augmentations, Inductive Bias, and Generalizationcited by 40
  3. 2023Nonlinear Meta-Learning Can Guarantee Faster Ratescited by 8
  4. 2024Random features models: a way to study the success of naive imputationcited by 7
  5. 2025Comparing regularisation paths of (conjugate) gradient estimators in ridge regressioncited by 2
  6. 2025Kernel Density Machinescited by 2
  7. 2025Fast kernel methods: Sobolev, physics-informed, and additive modelscited by 2
  8. 2023How many samples are needed to leverage smoothness?cited by 1
  9. 2024Incentivized Exploration with Stochastic Covariates: A Two-Stage Mechanism Design for Recommender Systemcited by 1
  10. 2025Enhancing Federated Class-Incremental Learning via Spatial-Temporal Statistics Aggregationcited by 1
  11. 2025Generalisation and benign over-fitting for linear regression onto random functional covariatescited by 1
  12. 2026Handling Covariate Mismatch in Collaborative Linear Predictioncited by 0
3 more
  1. 2026Filter Learning for Subgraphs: Algebras and Performance Risk Boundscited by 0
  2. 2026A Simple Approximation to the Distribution of the Ridge Regression Estimatorcited by 0
  3. 2026Stochastic gradient descent with initial regularizationcited by 0