# Multi-modal Bayesian Neural Network Surrogates with Conjugate Last-Layer Estimation

arXiv: https://arxiv.org/abs/2509.21711 · HTML: https://mlsanity.com/paper/2509.21711/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 58 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Builds on (most cited first, 20 of 20)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [Multimodal Deep Learning](https://mlsanity.com/paper/2301.04856/) (2023)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [Variational Bayesian Last Layers](https://mlsanity.com/paper/2404.11599/) (2024)
- [Greed is Good: Exploration and Exploitation Trade-offs in Bayesian Optimisation](https://mlsanity.com/paper/1911.12809/) (2019)
- [A Study of Bayesian Neural Network Surrogates for Bayesian Optimization](https://mlsanity.com/paper/2305.20028/) (2023)
- [Multi-Fidelity Bayesian Optimization via Deep Neural Networks](https://mlsanity.com/paper/2007.03117/) (2020)
- [Theoretical and Computational Guarantees of Mean Field Variational Inference for Community Detection](https://mlsanity.com/paper/1710.11268/) (2017)
- [Deep Multi-fidelity Gaussian Processes](https://mlsanity.com/paper/1604.07484/) (2016)
- [Unpaired Image-to-Speech Synthesis with Multimodal Information Bottleneck](https://mlsanity.com/paper/1908.07094/) (2019)
- [Bayesian Optimization via Continual Variational Last Layer Training](https://mlsanity.com/paper/2412.09477/) (2024)
- [On the role of Model Uncertainties in Bayesian Optimization](https://mlsanity.com/paper/2301.05983/) (2023)
- [Stability of Mean-Field Variational Inference](https://mlsanity.com/paper/2506.07856/) (2025)

## Abstract (arXiv)

As data collection and simulation capabilities advance, multi-modal learning, the task of learning from multiple modalities and sources of data, is becoming an increasingly important area of research. Surrogate models that learn from data of multiple auxiliary modalities to support the modeling of a highly expensive quantity of interest have the potential to aid outer loop applications such as optimization, inverse problems, or sensitivity analyses when multi-modal data are available. We develop two multi-modal Bayesian neural network surrogate models and leverage conditionally conjugate distributions in the last layer to estimate model parameters using stochastic variational inference (SVI). We provide a method to perform this conjugate SVI estimation in the presence of partially missing observations. We demonstrate improved prediction accuracy and uncertainty quantification compared to uni-modal surrogate models for both scalar and time series data.