# Iterative Methods for Vecchia-Laplace Approximations for Latent Gaussian Process Models

arXiv: https://arxiv.org/abs/2310.12000 · HTML: https://mlsanity.com/paper/2310.12000/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 18 Oct 2023.

## Builds on (most cited first, 17 of 17)

- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets](https://mlsanity.com/paper/1406.7343/) (2014)
- [A general framework for Vecchia approximations of Gaussian processes](https://mlsanity.com/paper/1708.06302/) (2017)
- [Constant-Time Predictive Distributions for Gaussian Processes](https://mlsanity.com/paper/1803.06058/) (2018)
- [A Case Study Competition Among Methods for Analyzing Large Spatial Data](https://mlsanity.com/paper/1710.05013/) (2017)
- [Sparse Cholesky factorization by Kullback-Leibler minimization](https://mlsanity.com/paper/2004.14455/) (2020)
- [Scalable Log Determinants for Gaussian Process Kernel Learning](https://mlsanity.com/paper/1711.03481/) (2017)
- [Gaussian Process Boosting](https://mlsanity.com/paper/2004.02653/) (2020)
- [Permutation and Grouping Methods for Sharpening Gaussian Process Approximations](https://mlsanity.com/paper/1609.05372/) (2016)
- [Preconditioning for Scalable Gaussian Process Hyperparameter Optimization](https://mlsanity.com/paper/2107.00243/) (2021)
- [Vecchia approximations of Gaussian-process predictions](https://mlsanity.com/paper/1805.03309/) (2018)
- [When are Iterative Gaussian Processes Reliably Accurate?](https://mlsanity.com/paper/2112.15246/) (2021)
- [Gaussian Process Learning via Fisher Scoring of Vecchia's Approximation](https://mlsanity.com/paper/1905.08374/) (2019)
- [Latent Gaussian Model Boosting](https://mlsanity.com/paper/2105.08966/) (2021)
- [Prior-preconditioned conjugate gradient method for accelerated Gibbs sampling in "large n & large p" Bayesian sparse regression](https://mlsanity.com/paper/1810.12437/) (2018)
- [Vecchia-Laplace approximations of generalized Gaussian processes for big non-Gaussian spatial data](https://mlsanity.com/paper/1906.07828/) (2019)
- [Kryging: Geostatistical analysis of large-scale datasets using Krylov subspace methods](https://mlsanity.com/paper/2012.13133/) (2020)

## Built on by (most cited first, 6 of 6)

- [Iterative Methods for Full-Scale Gaussian Process Approximations for Large Spatial Data](https://mlsanity.com/paper/2405.14492/) (2024)
- [A Spatio-Temporal Machine Learning Model for Mortgage Credit Risk: Default Probabilities and Loan Portfolios](https://mlsanity.com/paper/2410.02846/) (2024)
- [Scalable Krylov Subspace Methods for Generalized Mixed-Effects Models with Crossed Random Effects](https://mlsanity.com/paper/2505.09552/) (2025)
- [Vecchia-Inducing-Points Full-Scale Approximations for Gaussian Processes](https://mlsanity.com/paper/2507.05064/) (2025)
- [A Censored Transformed Model for Proportional Outcomes with Boundary Mass and an Application to Loss Given Default Modeling](https://mlsanity.com/paper/2606.21515/) (2026)
- [Adaptive KappaSharp: Condition-Number Shaping for Preferential Bayesian Optimization](https://mlsanity.com/paper/2608.07859/) (2026)

## Abstract (arXiv)

Latent Gaussian process (GP) models are flexible probabilistic non-parametric function models. Vecchia approximations are accurate approximations for GPs to overcome computational bottlenecks for large data, and the Laplace approximation is a fast method with asymptotic convergence guarantees to approximate marginal likelihoods and posterior predictive distributions for non-Gaussian likelihoods. Unfortunately, the computational complexity of combined Vecchia-Laplace approximations grows faster than linearly in the sample size when used in combination with direct solver methods such as the Cholesky decomposition. Computations with Vecchia-Laplace approximations can thus become prohibitively slow precisely when the approximations are usually the most accurate, i.e., on large data sets. In this article, we present iterative methods to overcome this drawback. Among other things, we introduce and analyze several preconditioners, derive new convergence results, and propose novel methods for accurately approximating predictive variances. We analyze our proposed methods theoretically and in experiments with simulated and real-world data. In particular, we obtain a speed-up of an order of magnitude compared to Cholesky-based calculations and a threefold increase in prediction accuracy in terms of the continuous ranked probability score compared to a state-of-the-art method on a large satellite data set. All methods are implemented in a free C++ software library with high-level Python and R packages.