# Fast calculation of Gaussian Process multiple-fold cross-validation residuals and their covariances

arXiv: https://arxiv.org/abs/2101.03108 · HTML: https://mlsanity.com/paper/2101.03108/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 35 references that are in the map.
- First posted on arXiv 8 Jan 2021.

## Builds on (most cited first, 6 of 6)

- [Learning dynamical systems from data: a simple cross-validation perspective](https://mlsanity.com/paper/2007.05074/) (2020)
- [Cross-Validation for Correlated Data](https://mlsanity.com/paper/1904.02438/) (2019)
- [Towards new cross-validation-based estimators for Gaussian process regression: efficient adjoint computation of gradients](https://mlsanity.com/paper/2002.11543/) (2020)
- [Model Selection for Gaussian Process Regression by Approximation Set Coding](https://mlsanity.com/paper/1610.00907/) (2016)
- [Assessing Prediction Error at Interpolation and Extrapolation Points](https://mlsanity.com/paper/1802.00996/) (2018)
- [Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data](https://mlsanity.com/paper/2001.00980/) (2020)

## Built on by (most cited first, 5 of 5)

- [Gaussian Process Surrogate Models for Neural Networks](https://mlsanity.com/paper/2208.06028/) (2022)
- [Distributional encoding for Gaussian process regression with qualitative inputs](https://mlsanity.com/paper/2506.04813/) (2025)
- [Weighted Leave-One-Out Cross Validation](https://mlsanity.com/paper/2505.19737/) (2025)
- [On the Usage of Gaussian Process for Efficient Data Valuation](https://mlsanity.com/paper/2506.04026/) (2025)
- [What Can a Gaussian Process Design Test](https://mlsanity.com/paper/2610.10122/) (2026)

## Abstract (arXiv)

We generalize fast Gaussian process leave-one-out formulae to multiple-fold cross-validation, highlighting in turn the covariance structure of cross-validation residuals in both Simple and Universal Kriging frameworks. We illustrate how resulting covariances affect model diagnostics. We further establish in the case of noiseless observations that correcting for covariances between residuals in cross-validation-based estimation of the scale parameter leads back to MLE. Also, we highlight in broader settings how differences between pseudo-likelihood and likelihood methods boil down to accounting or not for residual covariances. The proposed fast calculation of cross-validation residuals is implemented and benchmarked against a naive implementation. Numerical experiments highlight the accuracy and substantial speed-ups that our approach enables. However, as supported by a discussion on main drivers of computational costs and by a numerical benchmark, speed-ups steeply decline as the number of folds (say, all sharing the same size) decreases. An application to a contaminant localization test case illustrates that grouping clustered observations in folds may help improving model assessment and parameter fitting compared to Leave-One-Out. Overall, our results enable fast multiple-fold cross-validation, have direct consequences in model diagnostics, and pave the way to future work on hyperparameter fitting and on the promising field of goal-oriented fold design.