# Bayesian Surrogate Training on Multiple Data Sources: A Hybrid Modeling Strategy

arXiv: https://arxiv.org/abs/2412.11875 · HTML: https://mlsanity.com/paper/2412.11875/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 53 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 20 of 20)

- [Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks](https://mlsanity.com/paper/2002.10118/) (2020)
- [A General Framework for Updating Belief Distributions](https://mlsanity.com/paper/1306.6430/) (2013)
- [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](https://mlsanity.com/paper/1507.04544/) (2015)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [Variational Bayesian Last Layers](https://mlsanity.com/paper/2404.11599/) (2024)
- [Assigning a value to a power likelihood in a general Bayesian model](https://mlsanity.com/paper/1701.08515/) (2017)
- [Using stacking to average Bayesian predictive distributions](https://mlsanity.com/paper/1704.02030/) (2017)
- [Probabilistic Solutions To Ordinary Differential Equations As Non-Linear Bayesian Filtering: A New Perspective](https://mlsanity.com/paper/1810.03440/) (2018)
- [Data-driven polynomial chaos expansion for machine learning regression](https://mlsanity.com/paper/1808.03216/) (2018)
- [Calibrated Adaptive Probabilistic ODE Solvers](https://mlsanity.com/paper/2012.08202/) (2020)
- [A Probabilistic State Space Model for Joint Inference from Differential Equations and Data](https://mlsanity.com/paper/2103.10153/) (2021)
- [A SIDARTHE Model of COVID-19 Epidemic in Italy](https://mlsanity.com/paper/2003.09861/) (2020)
- [Improved uncertainty quantification for neural networks with Bayesian last layer](https://mlsanity.com/paper/2302.10975/) (2023)
- [Bayesian Surrogate Analysis and Uncertainty Propagation](https://mlsanity.com/paper/2101.04038/) (2021)
- [Detecting and diagnosing prior and likelihood sensitivity with power-scaling](https://mlsanity.com/paper/2107.14054/) (2021)
- [A fully Bayesian sparse polynomial chaos expansion approach with joint priors on the coefficients and global selection of terms](https://mlsanity.com/paper/2204.06043/) (2022)
- [Regression-based sparse polynomial chaos for uncertainty quantification of subsurface flow models](https://mlsanity.com/paper/1909.01648/) (2019)
- [Semi-Modular Inference: enhanced learning in multi-modular models by tempering the influence of components](https://mlsanity.com/paper/2003.06804/) (2020)
- [Uncertainty Quantification and Propagation in Surrogate-based Bayesian Inference](https://mlsanity.com/paper/2312.05153/) (2023)
- [Enhancing Polynomial Chaos Expansion Based Surrogate Modeling using a Novel Probabilistic Transfer Learning Strategy](https://mlsanity.com/paper/2312.04648/) (2023)

## Abstract (arXiv)

Surrogate models are often used as computationally efficient approximations to complex simulation models, enabling tasks such as solving inverse problems, sensitivity analysis, and probabilistic forward predictions, which would otherwise be computationally infeasible. During training, surrogate parameters are fitted such that the surrogate reproduces the simulation model's outputs as closely as possible. However, the simulation model itself is merely a simplification of the real-world system, often missing relevant processes or suffering from misspecifications e.g., in inputs or boundary conditions. Hints about these might be captured in real-world measurement data, and yet, we typically ignore those hints during surrogate building. In this paper, we propose two novel probabilistic approaches to integrate simulation data and real-world measurement data during surrogate training. The first method trains separate surrogate models for each data source and combines their predictive distributions, while the second incorporates both data sources by training a single surrogate. Both hybrid modeling approaches employ a novel weighting strategy for combining heterogeneous data sources during surrogate training, which operates independently of the chosen surrogate family. We show the conceptual differences and benefits of the two approaches through both synthetic and real-world case studies. The results demonstrate the potential of these methods to improve predictive accuracy, predictive coverage, and to diagnose problems in the underlying simulation model. These insights can improve system understanding and future model development.