# Recursive Estimation for Sparse Gaussian Process Regression

arXiv: https://arxiv.org/abs/1905.11711 · HTML: https://mlsanity.com/paper/1905.11711/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2019.

## Built on by (most cited first, 11 of 11)

- [Exact Gaussian Processes for Massive Datasets via Non-Stationary Sparsity-Discovering Kernels](https://mlsanity.com/paper/2205.09070/) (2022)
- [Correlated Product of Experts for Sparse Gaussian Process Regression](https://mlsanity.com/paper/2112.09519/) (2021)
- [Learning Choice Functions with Gaussian Processes](https://mlsanity.com/paper/2302.00406/) (2023)
- [Whom to Trust? Elective Learning for Distributed Gaussian Process Regression](https://mlsanity.com/paper/2402.03014/) (2024)
- [Modeling Complex Disease Trajectories using Deep Generative Models with Semi-Supervised Latent Processes](https://mlsanity.com/paper/2311.08149/) (2023)
- [Towards AI-Based Precision Oncology: A Machine Learning Framework for Personalized Counterfactual Treatment Suggestions based on Multi-Omics Data](https://mlsanity.com/paper/2402.12190/) (2024)
- [Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect](https://mlsanity.com/paper/2109.13891/) (2021)
- [Informative Planning for Worst-Case Error Minimisation in Sparse Gaussian Process Regression](https://mlsanity.com/paper/2203.03828/) (2022)
- [Semi-Supervised Generative Models for Disease Trajectories: A Case Study on Systemic Sclerosis](https://mlsanity.com/paper/2407.11427/) (2024)
- [Tensor network square root Kalman filter for online Gaussian process regression](https://mlsanity.com/paper/2409.03276/) (2024)
- [Comparison of Data Reduction Criteria for Online Gaussian Processes](https://mlsanity.com/paper/2508.10815/) (2025)

## Abstract (arXiv)

Gaussian Processes (GPs) are powerful kernelized methods for non-parameteric regression used in many applications. However, their use is limited to a few thousand of training samples due to their cubic time complexity. In order to scale GPs to larger datasets, several sparse approximations based on so-called inducing points have been proposed in the literature. In this work we investigate the connection between a general class of sparse inducing point GP regression methods and Bayesian recursive estimation which enables Kalman Filter like updating for online learning. The majority of previous work has focused on the batch setting, in particular for learning the model parameters and the position of the inducing points, here instead we focus on training with mini-batches. By exploiting the Kalman filter formulation, we propose a novel approach that estimates such parameters by recursively propagating the analytical gradients of the posterior over mini-batches of the data. Compared to state of the art methods, our method keeps analytic updates for the mean and covariance of the posterior, thus reducing drastically the size of the optimization problem. We show that our method achieves faster convergence and superior performance compared to state of the art sequential Gaussian Process regression on synthetic GP as well as real-world data with up to a million of data samples.