# Simultaneous analysis of approximate leave-one-out cross-validation and mean-field inference

arXiv: https://arxiv.org/abs/2501.02624 · HTML: https://mlsanity.com/paper/2501.02624/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 5 Jan 2025.

## Builds on (most cited first, 15 of 15)

- [Learning curves of generic features maps for realistic datasets with a teacher-student model](https://mlsanity.com/paper/2102.08127/) (2021)
- [A framework to characterize performance of LASSO algorithms](https://mlsanity.com/paper/1303.7291/) (2013)
- [The Lasso with general Gaussian designs with applications to hypothesis testing](https://mlsanity.com/paper/2007.13716/) (2020)
- [Approximate Leave-One-Out for Fast Parameter Tuning in High Dimensions](https://mlsanity.com/paper/1807.02694/) (2018)
- [Consistent Risk Estimation in Moderately High-Dimensional Linear Regression](https://mlsanity.com/paper/1902.01753/) (2019)
- [De-Biasing The Lasso With Degrees-of-Freedom Adjustment](https://mlsanity.com/paper/1902.08885/) (2019)
- [Tight Query Complexity Lower Bounds for PCA via Finite Sample Deformed Wigner Law](https://mlsanity.com/paper/1804.01221/) (2018)
- [Error bounds in estimating the out-of-sample prediction error using leave-one-out cross validation in high-dimensions](https://mlsanity.com/paper/2003.01770/) (2020)
- [Derivatives and residual distribution of regularized M-estimators with application to adaptive tuning](https://mlsanity.com/paper/2107.05143/) (2021)
- [Approximate Leave-One-Out for High-Dimensional Non-Differentiable Learning Problems](https://mlsanity.com/paper/1810.02716/) (2018)
- [CAD: Debiasing the Lasso with inaccurate covariate model](https://mlsanity.com/paper/2107.14172/) (2021)
- [Observable adjustments in single-index models for regularized M-estimators](https://mlsanity.com/paper/2204.06990/) (2022)
- [Challenges of the inconsistency regime: Novel debiasing methods for missing data models](https://mlsanity.com/paper/2309.01362/) (2023)
- [Theoretical Analysis of Leave-one-out Cross Validation for Non-differentiable Penalties under High-dimensional Settings](https://mlsanity.com/paper/2402.08543/) (2024)
- [Precise Asymptotics of Bagging Regularized M-estimators](https://mlsanity.com/paper/2409.15252/) (2024)

## Built on by (most cited first, 1 of 1)

- [Fast Leave-One-Out Approximation from Fragment-Target Prevalence Vectors (molFTP) : From Dummy Masking to Key-LOO for Leakage-Free Feature Construction](https://mlsanity.com/paper/2510.06029/) (2025)

## Abstract (arXiv)

Approximate Leave-One-Out Cross-Validation (ALO-CV) is a method that has been proposed to estimate the generalization error of a regularized estimator in the high-dimensional regime where dimension and sample size are of the same order, the so-called ``proportional regime''. A new analysis is developed to derive the consistency of ALO-CV for non-differentiable regularizers under Gaussian covariates and strong convexity. Using a conditioning argument, the difference between the ALO-CV weights and their counterparts in mean-field inference is shown to be small. Combined with upper bounds between the mean-field inference estimate and the leave-one-out quantity, this provides a proof that ALO-CV approximates the leave-one-out quantity up to negligible error terms. Linear models with square loss, robust linear regression and single-index models are explicitly treated.