# Hierarchical Inducing Point Gaussian Process for Inter-domain Observations

arXiv: https://arxiv.org/abs/2103.00393 · HTML: https://mlsanity.com/paper/2103.00393/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 41 references that are in the map.
- First posted on arXiv 28 Feb 2021.

## Builds on (most cited first, 16 of 16)

- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [Exact Gaussian Processes on a Million Data Points](https://mlsanity.com/paper/1903.08114/) (2019)
- [Stochastic Variational Deep Kernel Learning](https://mlsanity.com/paper/1611.00336/) (2016)
- [Understanding Probabilistic Sparse Gaussian Process Approximations](https://mlsanity.com/paper/1606.04820/) (2016)
- [A Framework for Interdomain and Multioutput Gaussian Processes](https://mlsanity.com/paper/2003.01115/) (2020)
- [MCMC for Variationally Sparse Gaussian Processes](https://mlsanity.com/paper/1506.04000/) (2015)
- [Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Models](https://mlsanity.com/paper/1803.09151/) (2018)
- [Preconditioning Kernel Matrices](https://mlsanity.com/paper/1602.06693/) (2016)
- [Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition](https://mlsanity.com/paper/1710.07324/) (2017)
- [Thoughts on Massively Scalable Gaussian Processes](https://mlsanity.com/paper/1511.01870/) (2015)
- [Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization](https://mlsanity.com/paper/2006.11267/) (2020)
- [Correcting boundary over-exploration deficiencies in Bayesian optimization with virtual derivative sign observations](https://mlsanity.com/paper/1704.00963/) (2017)
- [Scalable Gaussian Processes with Grid-Structured Eigenfunctions (GP-GRIEF)](https://mlsanity.com/paper/1807.02125/) (2018)
- [Evaluating the squared-exponential covariance function in Gaussian processes with integral observations](https://mlsanity.com/paper/1812.07319/) (2018)
- [On the construction of probabilistic Newton-type algorithms](https://mlsanity.com/paper/1704.01382/) (2017)

## Built on by (most cited first, 6 of 6)

- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Latent Graph Inference using Product Manifolds](https://mlsanity.com/paper/2211.16199/) (2022)
- [Vector-valued Gaussian Processes on Riemannian Manifolds via Gauge Independent Projected Kernels](https://mlsanity.com/paper/2110.14423/) (2021)
- [Variational Nearest Neighbor Gaussian Process](https://mlsanity.com/paper/2202.01694/) (2022)
- [Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processes](https://mlsanity.com/paper/2309.12658/) (2023)
- [B'ezier Gaussian Processes for Tall and Wide Data](https://mlsanity.com/paper/2209.00343/) (2022)

## Abstract (arXiv)

We examine the general problem of inter-domain Gaussian Processes (GPs): problems where the GP realization and the noisy observations of that realization lie on different domains. When the mapping between those domains is linear, such as integration or differentiation, inference is still closed form. However, many of the scaling and approximation techniques that our community has developed do not apply to this setting. In this work, we introduce the hierarchical inducing point GP (HIP-GP), a scalable inter-domain GP inference method that enables us to improve the approximation accuracy by increasing the number of inducing points to the millions. HIP-GP, which relies on inducing points with grid structure and a stationary kernel assumption, is suitable for low-dimensional problems. In developing HIP-GP, we introduce (1) a fast whitening strategy, and (2) a novel preconditioner for conjugate gradients which can be helpful in general GP settings. Our code is available at https: //github.com/cunningham-lab/hipgp.