# Approximate Inference for Fully Bayesian Gaussian Process Regression

arXiv: https://arxiv.org/abs/1912.13440 · HTML: https://mlsanity.com/paper/1912.13440/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2019.

## Built on by (most cited first, 30 of 31)

- [The Promises and Pitfalls of Deep Kernel Learning](https://mlsanity.com/paper/2102.12108/) (2021)
- [Neural Diffusion Processes](https://mlsanity.com/paper/2206.03992/) (2022)
- [Bayesian Active Learning with Fully Bayesian Gaussian Processes](https://mlsanity.com/paper/2205.10186/) (2022)
- [Self-Correcting Bayesian Optimization through Bayesian Active Learning](https://mlsanity.com/paper/2304.11005/) (2023)
- [Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inference](https://mlsanity.com/paper/2407.17033/) (2024)
- [Meta-learning Control Variates: Variance Reduction with Limited Data](https://mlsanity.com/paper/2303.04756/) (2023)
- [Adaptive Batch Sizes for Active Learning A Probabilistic Numerics Approach](https://mlsanity.com/paper/2306.05843/) (2023)
- [How Bayesian Should Bayesian Optimisation Be?](https://mlsanity.com/paper/2105.00894/) (2021)
- [Bayesian Optimisation for Active Monitoring of Air Pollution](https://mlsanity.com/paper/2202.07595/) (2022)
- [Gaussian Process Surrogate Models for Neural Networks](https://mlsanity.com/paper/2208.06028/) (2022)
- [Phi-DVAE: Physics-Informed Dynamical Variational Autoencoders for Unstructured Data Assimilation](https://mlsanity.com/paper/2209.15609/) (2022)
- [Leveraging Locality and Robustness to Achieve Massively Scalable Gaussian Process Regression](https://mlsanity.com/paper/2306.14731/) (2023)
- [Stochastic stiffness identification and response estimation of Timoshenko beams via physics-informed Gaussian processes](https://mlsanity.com/paper/2309.11875/) (2023)
- [Fully Bayesian inference for latent variable Gaussian process models](https://mlsanity.com/paper/2211.02218/) (2022)
- [Sparse Gaussian Process Hyperparameters: Optimize or Integrate?](https://mlsanity.com/paper/2211.02476/) (2022)
- [Physics-informed Gaussian process model for Euler-Bernoulli beam elements](https://mlsanity.com/paper/2308.02894/) (2023)
- [Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations](https://mlsanity.com/paper/2408.06069/) (2024)
- [BARK: A Fully Bayesian Tree Kernel for Black-box Optimization](https://mlsanity.com/paper/2503.05574/) (2025)
- [Informed Initialization for Bayesian Optimization and Active Learning](https://mlsanity.com/paper/2510.23681/) (2025)
- [Stochastic Cell Transmission Models of Traffic Networks](https://mlsanity.com/paper/2304.11654/) (2023)
- [Data-driven Prior Learning for Bayesian Optimisation](https://mlsanity.com/paper/2311.14653/) (2023)
- [Validating Gaussian Process Models with Simulation-Based Calibration](https://mlsanity.com/paper/2110.15049/) (2021)
- [A novel sampler for Gauss-Hermite determinantal point processes with application to Monte Carlo integration](https://mlsanity.com/paper/2203.08061/) (2022)
- [A Bayesian Approach To Graph Partitioning](https://mlsanity.com/paper/2204.12927/) (2022)
- [The case for fully Bayesian optimisation in small-sample trials](https://mlsanity.com/paper/2208.13960/) (2022)
- [Scalable Bayesian Transformed Gaussian Processes](https://mlsanity.com/paper/2210.10973/) (2022)
- [Applications of Gaussian Processes at Extreme Lengthscales: From Molecules to Black Holes](https://mlsanity.com/paper/2303.14291/) (2023)
- [Transfer Learning Bayesian Optimization to Design Competitor DNA Molecules for Use in Diagnostic Assays](https://mlsanity.com/paper/2402.17704/) (2024)
- [Robust Inference of Dynamic Covariance Using Wishart Processes and Sequential Monte Carlo](https://mlsanity.com/paper/2406.04796/) (2024)
- [Swine Diet Design using Multi-objective Regionalized Bayesian Optimization](https://mlsanity.com/paper/2409.12919/) (2024)

## Abstract (arXiv)

Learning in Gaussian Process models occurs through the adaptation of hyperparameters of the mean and the covariance function. The classical approach entails maximizing the marginal likelihood yielding fixed point estimates (an approach called textitType II maximum likelihood or ML-II). An alternative learning procedure is to infer the posterior over hyperparameters in a hierarchical specification of GPs we call textitFully Bayesian Gaussian Process Regression (GPR). This work considers two approximation schemes for the intractable hyperparameter posterior: 1) Hamiltonian Monte Carlo (HMC) yielding a sampling-based approximation and 2) Variational Inference (VI) where the posterior over hyperparameters is approximated by a factorized Gaussian (mean-field) or a full-rank Gaussian accounting for correlations between hyperparameters. We analyze the predictive performance for fully Bayesian GPR on a range of benchmark data sets.