# Variational sparse inverse Cholesky approximation for latent Gaussian processes via double Kullback-Leibler minimization

arXiv: https://arxiv.org/abs/2301.13303 · HTML: https://mlsanity.com/paper/2301.13303/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 46 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 19 of 19)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets](https://mlsanity.com/paper/1406.7343/) (2014)
- [A general framework for Vecchia approximations of Gaussian processes](https://mlsanity.com/paper/1708.06302/) (2017)
- [A Case Study Competition Among Methods for Analyzing Large Spatial Data](https://mlsanity.com/paper/1710.05013/) (2017)
- [Variational Gaussian Process State-Space Models](https://mlsanity.com/paper/1406.4905/) (2014)
- [Sparse Cholesky factorization by Kullback-Leibler minimization](https://mlsanity.com/paper/2004.14455/) (2020)
- [A Tutorial on Sparse Gaussian Processes and Variational Inference](https://mlsanity.com/paper/2012.13962/) (2020)
- [Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity](https://mlsanity.com/paper/1706.02205/) (2017)
- [Scaled Vecchia approximation for fast computer-model emulation](https://mlsanity.com/paper/2005.00386/) (2020)
- [Correlation-based sparse inverse Cholesky factorization for fast Gaussian-process inference](https://mlsanity.com/paper/2112.14591/) (2021)
- [Variational Nearest Neighbor Gaussian Process](https://mlsanity.com/paper/2202.01694/) (2022)
- [Scalable Gaussian-process regression and variable selection using Vecchia approximations](https://mlsanity.com/paper/2202.12981/) (2022)
- [Vecchia-Laplace approximations of generalized Gaussian processes for big non-Gaussian spatial data](https://mlsanity.com/paper/1906.07828/) (2019)
- [A Scalable Method to Exploit Screening in Gaussian Process Models with Noise](https://mlsanity.com/paper/2208.06877/) (2022)
- [Inducing Gaussian Process Networks](https://mlsanity.com/paper/2204.09889/) (2022)

## Built on by (most cited first, 4 of 4)

- [Preventing Model Collapse in Gaussian Process Latent Variable Models](https://mlsanity.com/paper/2404.01697/) (2024)
- [Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling](https://mlsanity.com/paper/2505.16481/) (2025)
- [Vecchia-Inducing-Points Full-Scale Approximations for Gaussian Processes](https://mlsanity.com/paper/2507.05064/) (2025)
- [Scalable Derivative Gaussian Processes via Exact Gradient Reduction](https://mlsanity.com/paper/2606.02909/) (2026)

## Abstract (arXiv)

To achieve scalable and accurate inference for latent Gaussian processes, we propose a variational approximation based on a family of Gaussian distributions whose covariance matrices have sparse inverse Cholesky (SIC) factors. We combine this variational approximation of the posterior with a similar and efficient SIC-restricted Kullback-Leibler-optimal approximation of the prior. We then focus on a particular SIC ordering and nearest-neighbor-based sparsity pattern resulting in highly accurate prior and posterior approximations. For this setting, our variational approximation can be computed via stochastic gradient descent in polylogarithmic time per iteration. We provide numerical comparisons showing that the proposed double-Kullback-Leibler-optimal Gaussian-process approximation (DKLGP) can sometimes be vastly more accurate for stationary kernels than alternative approaches such as inducing-point and mean-field approximations at similar computational complexity.