mlsanityData to 7 Oct 2026

Theoretical Analysis of Leave-one-out Cross Validation for Non-differentiable Penalties under High-dimensional Settings

arXiv 2402.08543 ↗First posted 13 Feb 2024math.ST stat.ML stat.THcited by 5

Not on the map. Why?

Abstract (arXiv)

Despite a large and significant body of recent work focused on estimating the out-of-sample risk of regularized models in the high dimensional regime, a theoretical understanding of this problem for non-differentiable penalties such as generalized LASSO and nuclear norm is missing. In this paper we resolve this challenge. We study this problem in the proportional high dimensional regime where both the sample size n and number of features p are large, and n/p and the signal-to-noise ratio (per observation) remain finite. We provide finite sample upper bounds on the expected squared error of leave-one-out cross-validation (LO) in estimating the out-of-sample risk. The theoretical framework presented here provides a solid foundation for elucidating empirical findings that show the accuracy of LO.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 16 of its 39 references that are in the map. First posted on arXiv 13 Feb 2024.

Builds on

16 of 39 references.

References · 16

  1. 2020A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-ℓ1-Norm Interpolated Classifierscited by 44
  2. 2020The Lasso with general Gaussian designs with applications to hypothesis testingcited by 40
  3. 2020Approximate Message Passing algorithms for rotationally invariant matricescited by 31
  4. 2022A Non-Asymptotic Framework for Approximate Message Passing in Spiked Modelscited by 18
  5. 2017Subset Selection with Shrinkage: Sparse Linear Modeling when the SNR is lowcited by 17
  6. 2017Extended Comparisons of Best Subset Selection, Forward Stepwise Selection, and the Lassocited by 16
  7. 2019Consistent Risk Estimation in Moderately High-Dimensional Linear Regressioncited by 14
  8. 2019Algorithmic Analysis and Statistical Estimation of SLOPE via Approximate Message Passingcited by 14
  9. 2017On Optimal Generalizability in Parametric Learningcited by 12
  10. 2020Error bounds in estimating the out-of-sample prediction error using leave-one-out cross validation in high-dimensionscited by 11
  11. 2019Does SLOPE outperform bridge regression?cited by 8
  12. 2021Can we globally optimize cross-validation loss? Quasiconvexity in ridge regressioncited by 8
4 more
  1. 2023Iterative Approximate Cross-Validationcited by 7
  2. 2015Consistent Parameter Estimation for LASSO and Approximate Message Passingcited by 6
  3. 2023Approximate Leave-one-out Cross Validation for Regression with ℓ1 Regularizers (extended version)cited by 3
  4. 2019Cross validation in sparse linear regression with piecewise continuous nonconvex penalties and its accelerationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Certified Data Removal Under High-dimensional Settingscited by 5
  2. 2025Gaussian Certified Unlearning in High Dimensions: A Hypothesis Testing Approachcited by 5
  3. 2025Simultaneous analysis of approximate leave-one-out cross-validation and mean-field inferencecited by 1
  4. 2025Evaluation and Optimization of Leave-one-out Cross-validation for the Lassocited by 0
  5. 2026Accelerating Conformal Prediction via Approximate Leave-One-Outcited by 0