# Practical Hilbert space approximate Bayesian Gaussian processes for probabilistic programming

arXiv: https://arxiv.org/abs/2004.11408 · HTML: https://mlsanity.com/paper/2004.11408/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2020.

## Built on by (most cited first, 28 of 28)

- [Pathfinder: Parallel quasi-Newton variational inference](https://mlsanity.com/paper/2108.03782/) (2021)
- [No-Regret Algorithms for Time-Varying Bayesian Optimization](https://mlsanity.com/paper/2102.06296/) (2021)
- [Random Exploration in Bayesian Optimization: Order-Optimal Regret and Computational Efficiency](https://mlsanity.com/paper/2310.15351/) (2023)
- [Improved Regret Bounds for Gaussian Process Upper Confidence Bound in Bayesian Optimization](https://mlsanity.com/paper/2506.01393/) (2025)
- [The GeometricKernels Package: Heat and Mat'ern Kernels for Geometric Learning on Manifolds, Meshes, and Graphs](https://mlsanity.com/paper/2407.08086/) (2024)
- [Epsilon-Greedy Thompson Sampling to Bayesian Optimization](https://mlsanity.com/paper/2403.00540/) (2024)
- [Efficient Exploration in Binary and Preferential Bayesian Optimization](https://mlsanity.com/paper/2110.09361/) (2021)
- [A fully Bayesian sparse polynomial chaos expansion approach with joint priors on the coefficients and global selection of terms](https://mlsanity.com/paper/2204.06043/) (2022)
- [Dynamic Online Ensembles of Basis Expansions](https://mlsanity.com/paper/2405.01365/) (2024)
- [Quantum-Assisted Hilbert-Space Gaussian Process Regression](https://mlsanity.com/paper/2402.00544/) (2024)
- [Differentially Private Kernelized Contextual Bandits](https://mlsanity.com/paper/2501.07046/) (2025)
- [Sampling from Gaussian Processes: A Tutorial and Applications in Global Sensitivity Analysis and Optimization](https://mlsanity.com/paper/2507.14746/) (2025)
- [Contextual Bayesian optimization with binary outputs](https://mlsanity.com/paper/2111.03447/) (2021)
- [Explainable Learning with Gaussian Processes](https://mlsanity.com/paper/2403.07072/) (2024)
- [Bayesian Inference and Learning in Nonlinear Dynamical Systems: A Framework for Incorporating Explicit and Implicit Prior Knowledge](https://mlsanity.com/paper/2508.15345/) (2025)
- [Learning Dynamics from Input-Output Data with Hamiltonian Gaussian Processes](https://mlsanity.com/paper/2511.05330/) (2025)
- [Nearly-Optimal Algorithm for Adversarial Kernelized Bandits](https://mlsanity.com/paper/2605.10299/) (2026)
- [Scalable mixed-domain Gaussian process modeling and model reduction for longitudinal data](https://mlsanity.com/paper/2111.02019/) (2021)
- [Push: Concurrent Probabilistic Programming for Bayesian Deep Learning](https://mlsanity.com/paper/2306.06528/) (2023)
- [Towards Multilevel Modelling of Train Passing Events on the Staffordshire Bridge](https://mlsanity.com/paper/2403.17820/) (2024)
- [Exploiting Hankel-Toeplitz Structures for Fast Computation of Kernel Precision Matrices](https://mlsanity.com/paper/2408.02346/) (2024)
- [Pareto-Optimal Anytime Algorithms via Bayesian Racing](https://mlsanity.com/paper/2603.08493/) (2026)
- [Time-adaptive infinite-dimensional Gaussian process regression on manifolds](https://mlsanity.com/paper/2603.21144/) (2026)
- [Fast and Provably Accurate Sequential Designs using Hilbert Space Gaussian Processes](https://mlsanity.com/paper/2604.20414/) (2026)
- [Differentiable latent structure discovery for interpretable forecasting in clinical time series](https://mlsanity.com/paper/2604.27967/) (2026)
- [Sharper Guarantees for Misspecified Kernelized Bandit Optimization](https://mlsanity.com/paper/2605.05967/) (2026)
- [Modeling Temporal scRNA-seq Data with Latent Gaussian Process and Optimal Transport](https://mlsanity.com/paper/2605.20989/) (2026)
- [On Basis Function Selection for Sparse Gaussian Process Regression](https://mlsanity.com/paper/2609.26624/) (2026)

## Abstract (arXiv)

Gaussian processes are powerful non-parametric probabilistic models for stochastic functions. However, the direct implementation entails a complexity that is computationally intractable when the number of observations is large, especially when estimated with fully Bayesian methods such as Markov chain Monte Carlo. In this paper, we focus on a low-rank approximate Bayesian Gaussian processes, based on a basis function approximation via Laplace eigenfunctions for stationary covariance functions. The main contribution of this paper is a detailed analysis of the performance, and practical recommendations for how to select the number of basis functions and the boundary factor. Intuitive visualizations and recommendations, make it easier for users to improve approximation accuracy and computational performance. We also propose diagnostics for checking that the number of basis functions and the boundary factor are adequate given the data. The approach is simple and exhibits an attractive computational complexity due to its linear structure, and it is easy to implement in probabilistic programming frameworks. Several illustrative examples of the performance and applicability of the method in the probabilistic programming language Stan are presented together with the underlying Stan model code.