mlsanityData to 14 Oct 2026

Subsample Ridge Ensembles: Equivalences and Generalized Cross-Validation

arXiv 2304.13016 ↗First posted 25 Apr 2023math.ST cs.LG stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

We study subsampling-based ridge ensembles in the proportional asymptotics regime, where the feature size grows proportionally with the sample size such that their ratio converges to a constant. By analyzing the squared prediction risk of ridge ensembles as a function of the explicit penalty λ and the limiting subsample aspect ratio φs (the ratio of the feature size to the subsample size), we characterize contours in the (λ, φs)-plane at any achievable risk. As a consequence, we prove that the risk of the optimal full ridgeless ensemble (fitted on all possible subsamples) matches that of the optimal ridge predictor. In addition, we prove strong uniform consistency of generalized cross-validation (GCV) over the subsample sizes for estimating the prediction risk of ridge ensembles. This allows for GCV-based tuning of full ridgeless ensembles without sample splitting and yields a predictor whose risk matches optimal ridge risk.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 19 of its 86 references that are in the map. First posted on arXiv 25 Apr 2023.

Builds on

19 of 86 references.

References · 19

  1. 2019Benign Overfitting in Linear Regressioncited by 549
  2. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 513
  3. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 413
  4. 2021Deep learning: a statistical viewpointcited by 240
  5. 2019Two models of double descent for weak featurescited by 195
  6. 2015High-Dimensional Asymptotics of Prediction: Ridge Regression and Classificationcited by 148
  7. 2019Harmless interpolation of noisy data in regressioncited by 121
  8. 2020The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalizationcited by 101
  9. 2020Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regimecited by 95
  10. 2022More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations Generalizecited by 91
  11. 2020Understanding Double Descent Requires a Fine-Grained Bias-Variance Decompositioncited by 76
  12. 2019The Implicit Regularization of Ordinary Least Squares Ensemblescited by 25
7 more
  1. 2018Distributed linear regression by averagingcited by 21
  2. 2020Ridge Regularization: an Essential Concept in Data Sciencecited by 19
  3. 2022Fluctuations, Bias, Variance & Ensemble of Learners: Exact Asymptotics for Convex Losses in High-Dimensioncited by 19
  4. 2018Approximate Leave-One-Out for Fast Parameter Tuning in High Dimensionscited by 15
  5. 2020Error bounds in estimating the out-of-sample prediction error using leave-one-out cross validation in high-dimensionscited by 11
  6. 2022Bagging in overparameterized learning: Risk characterization and risk monotonizationcited by 11
  7. 2022Mitigating multiple descents: A model-agnostic framework for risk monotonizationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 2; 2026: 1.

11 papers.

  1. 2023Asymptotically free sketched ridge ensembles: Risks, cross-validation, and tuningcited by 11
  2. 2023Generalized equivalences between subsampling and ridge regularizationcited by 10
  3. 2023Corrected generalized cross-validation for finite ensembles of penalized estimatorscited by 5
  4. 2024Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regressioncited by 4
  5. 2024Failures and Successes of Cross-Validation for Early-Stopped Gradient Descentcited by 4
  6. 2024A replica analysis of under-baggingcited by 4
  7. 2024Revisiting Optimism and Model Complexity in the Wake of Overparameterized Machine Learningcited by 2
  8. 2024Implicit Regularization Paths of Weighted Neural Representationscited by 1
  9. 2025Revisiting Randomization in Greedy Model Searchcited by 1
  10. 2025Mallows-type model averaging: Non-asymptotic analysis and all-subset combinationcited by 0
  11. 2026DIAL: Position-Debiased LLM Judges with Adaptive Human Preference Calibrationcited by 0