# BITS for GAPS: Bayesian Information-Theoretic Sampling for hierarchical GAussian Process Surrogates

arXiv: https://arxiv.org/abs/2511.16815 · HTML: https://mlsanity.com/paper/2511.16815/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 93 references that are in the map.
- First posted on arXiv 20 Nov 2025.

## Builds on (most cited first, 9 of 9)

- [GPflow: A Gaussian process library using TensorFlow](https://mlsanity.com/paper/1610.08733/) (2016)
- [Approximate Inference for Fully Bayesian Gaussian Process Regression](https://mlsanity.com/paper/1912.13440/) (2019)
- [COBALT: COnstrained Bayesian optimizAtion of computationaLly expensive grey-box models exploiting derivaTive information](https://mlsanity.com/paper/2105.04114/) (2021)
- [Combining Multi-Fidelity Modelling and Asynchronous Batch Bayesian Optimization](https://mlsanity.com/paper/2211.06149/) (2022)
- [New Paradigms for Exploiting Parallel Experiments in Bayesian Optimization](https://mlsanity.com/paper/2210.01071/) (2022)
- [No-Regret Constrained Bayesian Optimization of Noisy and Expensive Hybrid Models using Differentiable Quantile Function Approximations](https://mlsanity.com/paper/2305.03824/) (2023)
- [Introducing Hybrid Modeling with Time-series-Transformers: A Comparative Study of Series and Parallel Approach in Batch Crystallization](https://mlsanity.com/paper/2308.05749/) (2023)
- [Human-Algorithm Collaborative Bayesian Optimization for Engineering Systems](https://mlsanity.com/paper/2404.10949/) (2024)
- [BO4IO: A Bayesian optimization approach to inverse optimization with uncertainty quantification](https://mlsanity.com/paper/2405.17875/) (2024)

## Abstract (arXiv)

We introduce Bayesian Information-Theoretic Sampling for hierarchical GAussian Process Surrogates (BITS for GAPS), a framework enabling information-theoretic experimental design of Gaussian process-based surrogate models. Unlike standard methods, which use fixed or point-estimated hyperparameters in acquisition functions, our approach propagates hyperparameter uncertainty into the sampling criterion through Bayesian hierarchical modeling. In this framework, a latent function receives a Gaussian process prior, while hyperparameters are assigned additional priors to capture the modeler's knowledge of the governing physical phenomena. Consequently, the acquisition function incorporates uncertainties from both the latent function and its hyperparameters, ensuring that sampling is guided by both data scarcity and model uncertainty. We further establish theoretical results in this context: a closed-form approximation and a lower bound of the posterior differential entropy. We demonstrate the framework's utility for hybrid modeling with a vapor-liquid equilibrium case study. Specifically, we build a surrogate model for latent activity coefficients in a binary mixture. We construct a hybrid model by embedding the surrogate into an extended form of Raoult's law. This hybrid model then informs distillation design. This case study shows how partial physical knowledge can be translated into a hierarchical Gaussian process surrogate. It also shows that using BITS for GAPS increases expected information gain and predictive accuracy by targeting high-uncertainty regions of the Wilson activity model. Overall, BITS for GAPS is a generalized uncertainty-aware framework for adaptive data acquisition in complex physical systems.