# Variational Inference for Gaussian Process Models with Linear Complexity

arXiv: https://arxiv.org/abs/1711.10127 · HTML: https://mlsanity.com/paper/1711.10127/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2017.

## Built on by (most cited first, 21 of 21)

- [Generalized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learning](https://mlsanity.com/paper/2205.06342/) (2022)
- [Calibrating Transformers via Sparse Gaussian Processes](https://mlsanity.com/paper/2303.02444/) (2023)
- [Tighter Bounds on the Log Marginal Likelihood of Gaussian Process Regression Using Conjugate Gradients](https://mlsanity.com/paper/2102.08314/) (2021)
- [Dual Parameterization of Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2111.03412/) (2021)
- [Variational Linearized Laplace Approximation for Bayesian Deep Learning](https://mlsanity.com/paper/2302.12565/) (2023)
- [Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inference](https://mlsanity.com/paper/2407.17033/) (2024)
- [Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition](https://mlsanity.com/paper/2106.05992/) (2021)
- [Variational Nearest Neighbor Gaussian Process](https://mlsanity.com/paper/2202.01694/) (2022)
- [Input Dependent Sparse Gaussian Processes](https://mlsanity.com/paper/2107.07281/) (2021)
- [GP-ConvCNP: Better Generalization for Convolutional Conditional Neural Processes on Time Series Data](https://mlsanity.com/paper/2106.04967/) (2021)
- [Variational Gaussian Processes: A Functional Analysis View](https://mlsanity.com/paper/2110.12798/) (2021)
- [A Deterministic Sampling Method via Maximum Mean Discrepancy Flow with Adaptive Kernel](https://mlsanity.com/paper/2111.10722/) (2021)
- [Towards Reliable Uncertainty Quantification via Deep Ensembles in Multi-output Regression Task](https://mlsanity.com/paper/2303.16210/) (2023)
- [Fixed-Mean Gaussian Processes for Post-hoc Bayesian Deep Learning](https://mlsanity.com/paper/2412.04177/) (2024)
- [New Bounds for Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2502.08730/) (2025)
- [Gaussian Experts Selection using Graphical Models](https://mlsanity.com/paper/2102.01496/) (2021)
- [Spherical Inducing Features for Orthogonally-Decoupled Gaussian Processes](https://mlsanity.com/paper/2304.14034/) (2023)
- [Generative Particle Variational Inference via Estimation of Functional Gradients](https://mlsanity.com/paper/2103.01291/) (2021)
- [Understanding Uncertainty in Bayesian Deep Learning](https://mlsanity.com/paper/2106.13055/) (2021)
- [Latent structure blockmodels for Bayesian spectral graph clustering](https://mlsanity.com/paper/2107.01734/) (2021)
- [Entry Dependent Expert Selection in Distributed Gaussian Processes Using Multilabel Classification](https://mlsanity.com/paper/2211.09940/) (2022)

## Abstract (arXiv)

Large-scale Gaussian process inference has long faced practical challenges due to time and space complexity that is superlinear in dataset size. While sparse variational Gaussian process models are capable of learning from large-scale data, standard strategies for sparsifying the model can prevent the approximation of complex functions. In this work, we propose a novel variational Gaussian process model that decouples the representation of mean and covariance functions in reproducing kernel Hilbert space. We show that this new parametrization generalizes previous models. Furthermore, it yields a variational inference problem that can be solved by stochastic gradient ascent with time and space complexity that is only linear in the number of mean function parameters, regardless of the choice of kernels, likelihoods, and inducing points. This strategy makes the adoption of large-scale expressive Gaussian process models possible. We run several experiments on regression tasks and show that this decoupled approach greatly outperforms previous sparse variational Gaussian process inference procedures.