# Fully Bayesian inference for latent variable Gaussian process models

arXiv: https://arxiv.org/abs/2211.02218 · HTML: https://mlsanity.com/paper/2211.02218/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 41 references that are in the map.
- First posted on arXiv 4 Nov 2022.

## Builds on (most cited first, 10 of 10)

- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [Understanding Probabilistic Sparse Gaussian Process Approximations](https://mlsanity.com/paper/1606.04820/) (2016)
- [Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variables](https://mlsanity.com/paper/1910.01688/) (2019)
- [Approximate Inference for Fully Bayesian Gaussian Process Regression](https://mlsanity.com/paper/1912.13440/) (2019)
- [A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors](https://mlsanity.com/paper/1806.07504/) (2018)
- [Group kernels for Gaussian process metamodels with categorical inputs](https://mlsanity.com/paper/1802.02368/) (2018)
- [Bayesian optimization of variable-size design space problems](https://mlsanity.com/paper/2003.03300/) (2020)
- [Data-Centric Mixed-Variable Bayesian Optimization For Materials Design](https://mlsanity.com/paper/1907.02577/) (2019)

## Built on by (most cited first, 4 of 4)

- [Rapid Design of Top-Performing Metal-Organic Frameworks with Qualitative Representations of Building Blocks](https://mlsanity.com/paper/2302.09184/) (2023)
- [Interpretable Multi-Source Data Fusion Through Latent Variable Gaussian Process](https://mlsanity.com/paper/2402.04146/) (2024)
- [Emerging Microelectronic Materials by Design: Navigating Combinatorial Design Space with Scarce and Dispersed Data](https://mlsanity.com/paper/2412.17283/) (2024)
- [Fast Riemannian-manifold Hamiltonian Monte Carlo for hierarchical Gaussian-process models](https://mlsanity.com/paper/2511.06407/) (2025)

## Abstract (arXiv)

Real engineering and scientific applications often involve one or more qualitative inputs. Standard Gaussian processes (GPs), however, cannot directly accommodate qualitative inputs. The recently introduced latent variable Gaussian process (LVGP) overcomes this issue by first mapping each qualitative factor to underlying latent variables (LVs), and then uses any standard GP covariance function over these LVs. The LVs are estimated similarly to the other GP hyperparameters through maximum likelihood estimation, and then plugged into the prediction expressions. However, this plug-in approach will not account for uncertainty in estimation of the LVs, which can be significant especially with limited training data. In this work, we develop a fully Bayesian approach for the LVGP model and for visualizing the effects of the qualitative inputs via their LVs. We also develop approximations for scaling up LVGPs and fully Bayesian inference for the LVGP hyperparameters. We conduct numerical studies comparing plug-in inference against fully Bayesian inference over a few engineering models and material design applications. In contrast to previous studies on standard GP modeling that have largely concluded that a fully Bayesian treatment offers limited improvements, our results show that for LVGP modeling it offers significant improvements in prediction accuracy and uncertainty quantification over the plug-in approach.