# When Gaussian Process Meets Big Data: A Review of Scalable GPs

arXiv: https://arxiv.org/abs/1807.01065 · HTML: https://mlsanity.com/paper/1807.01065/

## Key facts

- Cited by 206 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2018.

## Built on by (most cited first, 30 of 206)

- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [Active Preference-Based Gaussian Process Regression for Reward Learning](https://mlsanity.com/paper/2005.02575/) (2020)
- [Variational Bayesian Last Layers](https://mlsanity.com/paper/2404.11599/) (2024)
- [Causal Inference for Time series Analysis: Problems, Methods and Evaluation](https://mlsanity.com/paper/2102.05829/) (2021)
- [A review on data-driven constitutive laws for solids](https://mlsanity.com/paper/2405.03658/) (2024)
- [Sparse Cholesky Factorization for Solving Nonlinear PDEs via Gaussian Processes](https://mlsanity.com/paper/2304.01294/) (2023)
- [Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial](https://mlsanity.com/paper/2305.04933/) (2023)
- [GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning](https://mlsanity.com/paper/2102.07868/) (2021)
- [Differentiable sampling of molecular geometries with uncertainty-based adversarial attacks](https://mlsanity.com/paper/2101.11588/) (2021)
- [Information Directed Reward Learning for Reinforcement Learning](https://mlsanity.com/paper/2102.12466/) (2021)
- [MeltpoolNet: Melt pool Characteristic Prediction in Metal Additive Manufacturing Using Machine Learning](https://mlsanity.com/paper/2201.11662/) (2022)
- [Improved Convergence Rates for Sparse Approximation Methods in Kernel-Based Learning](https://mlsanity.com/paper/2202.04005/) (2022)
- [Data-Driven Design for Metamaterials and Multiscale Systems: A Review](https://mlsanity.com/paper/2307.05506/) (2023)
- [Traffic Prediction using Artificial Intelligence: Review of Recent Advances and Emerging Opportunities](https://mlsanity.com/paper/2305.19591/) (2023)
- [Deep Learning for Bayesian Optimization of Scientific Problems with High-Dimensional Structure](https://mlsanity.com/paper/2104.11667/) (2021)
- [Interactive Segmentation as Gaussian Process Classification](https://mlsanity.com/paper/2302.14578/) (2023)
- [Stochastic Process Learning via Operator Flow Matching](https://mlsanity.com/paper/2501.04126/) (2025)
- [Uncertainty Quantification for Machine Learning in Healthcare: A Survey](https://mlsanity.com/paper/2505.02874/) (2025)
- [Adaptive machine learning for protein engineering](https://mlsanity.com/paper/2106.05466/) (2021)
- [Deep Gaussian Processes: A Survey](https://mlsanity.com/paper/2106.12135/) (2021)
- [Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes](https://mlsanity.com/paper/2109.09690/) (2021)
- [On the Robustness of Bayesian Neural Networks to Adversarial Attacks](https://mlsanity.com/paper/2207.06154/) (2022)
- [Improved uncertainty quantification for neural networks with Bayesian last layer](https://mlsanity.com/paper/2302.10975/) (2023)
- [Distributionally Robust Statistical Verification with Imprecise Neural Networks](https://mlsanity.com/paper/2308.14815/) (2023)
- [Automatic nonlinear MPC approximation with closed-loop guarantees](https://mlsanity.com/paper/2312.10199/) (2023)
- [Machine learning for modelling unstructured grid data in computational physics: a review](https://mlsanity.com/paper/2502.09346/) (2025)
- [Neural Process for Black-Box Model Optimization Under Bayesian Framework](https://mlsanity.com/paper/2104.02487/) (2021)
- [Scalable Gaussian Processes for Data-Driven Design using Big Data with Categorical Factors](https://mlsanity.com/paper/2106.15356/) (2021)
- [Non-separable Spatio-temporal Graph Kernels via SPDEs](https://mlsanity.com/paper/2111.08524/) (2021)
- [Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture](https://mlsanity.com/paper/2210.02676/) (2022)

## Abstract (arXiv)

The vast quantity of information brought by big data as well as the evolving computer hardware encourages success stories in the machine learning community. In the meanwhile, it poses challenges for the Gaussian process (GP) regression, a well-known non-parametric and interpretable Bayesian model, which suffers from cubic complexity to data size. To improve the scalability while retaining desirable prediction quality, a variety of scalable GPs have been presented. But they have not yet been comprehensively reviewed and analyzed in order to be well understood by both academia and industry. The review of scalable GPs in the GP community is timely and important due to the explosion of data size. To this end, this paper is devoted to the review on state-of-the-art scalable GPs involving two main categories: global approximations which distillate the entire data and local approximations which divide the data for subspace learning. Particularly, for global approximations, we mainly focus on sparse approximations comprising prior approximations which modify the prior but perform exact inference, posterior approximations which retain exact prior but perform approximate inference, and structured sparse approximations which exploit specific structures in kernel matrix; for local approximations, we highlight the mixture/product of experts that conducts model averaging from multiple local experts to boost predictions. To present a complete review, recent advances for improving the scalability and capability of scalable GPs are reviewed. Finally, the extensions and open issues regarding the implementation of scalable GPs in various scenarios are reviewed and discussed to inspire novel ideas for future research avenues.