# Probabilistic Neural Data Fusion for Learning from an Arbitrary Number of Multi-fidelity Data Sets

arXiv: https://arxiv.org/abs/2301.13271 · HTML: https://mlsanity.com/paper/2301.13271/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 60 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 19 of 19)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks](https://mlsanity.com/paper/2002.10118/) (2020)
- [On transfer learning of neural networks using bi-fidelity data for uncertainty propagation](https://mlsanity.com/paper/2002.04495/) (2020)
- [Latent Map Gaussian Processes for Mixed Variable Metamodeling](https://mlsanity.com/paper/2102.03935/) (2021)
- [Multi-Fidelity Cost-Aware Bayesian Optimization](https://mlsanity.com/paper/2211.02732/) (2022)
- [Data Fusion with Latent Map Gaussian Processes](https://mlsanity.com/paper/2112.02206/) (2021)
- [On the Validity of Bayesian Neural Networks for Uncertainty Estimation](https://mlsanity.com/paper/1912.01530/) (2019)
- [MFNets: Data efficient all-at-once learning of multifidelity surrogates as directed networks of information sources](https://mlsanity.com/paper/2008.02672/) (2020)
- [Calibrating a large computer experiment simulating radiative shock hydrodynamics](https://mlsanity.com/paper/1410.3293/) (2014)
- [Representing model inadequacy: A stochastic operator approach](https://mlsanity.com/paper/1604.01651/) (2016)
- [A Stochastic Operator Approach to Model Inadequacy with Applications to Contaminant Transport](https://mlsanity.com/paper/1702.07779/) (2017)
- [Embedded discrepancy operators in reduced models of interacting species](https://mlsanity.com/paper/1910.08191/) (2019)
- [Adaptive Spatiotemporal Dimension Reduction in Concurrent Multiscale Damage Analysis](https://mlsanity.com/paper/2205.12149/) (2022)
- [Data-Driven Calibration of Multi-Fidelity Multiscale Fracture Models via Latent Map Gaussian Process](https://mlsanity.com/paper/2205.12157/) (2022)

## Built on by (most cited first, 5 of 5)

- [A Framework for Strategic Discovery of Credible Neural Network Surrogate Models under Uncertainty](https://mlsanity.com/paper/2403.08901/) (2024)
- [Bayesian neural networks for predicting uncertainty in full-field material response](https://mlsanity.com/paper/2406.14838/) (2024)
- [On the Effects of Heterogeneous Errors on Multi-fidelity Bayesian Optimization](https://mlsanity.com/paper/2309.02771/) (2023)
- [Multi-fidelity Design of Porous Microstructures for Thermofluidic Applications](https://mlsanity.com/paper/2310.18505/) (2023)
- [Should We Simultaneously Calibrate Multiple Computer Models?](https://mlsanity.com/paper/2505.18176/) (2025)

## Abstract (arXiv)

In many applications in engineering and sciences analysts have simultaneous access to multiple data sources. In such cases, the overall cost of acquiring information can be reduced via data fusion or multi-fidelity (MF) modeling where one leverages inexpensive low-fidelity (LF) sources to reduce the reliance on expensive high-fidelity (HF) data. In this paper, we employ neural networks (NNs) for data fusion in scenarios where data is very scarce and obtained from an arbitrary number of sources with varying levels of fidelity and cost. We introduce a unique NN architecture that converts MF modeling into a nonlinear manifold learning problem. Our NN architecture inversely learns non-trivial (e.g., non-additive and non-hierarchical) biases of the LF sources in an interpretable and visualizable manifold where each data source is encoded via a low-dimensional distribution. This probabilistic manifold quantifies model form uncertainties such that LF sources with small bias are encoded close to the HF source. Additionally, we endow the output of our NN with a parametric distribution not only to quantify aleatoric uncertainties, but also to reformulate the network's loss function based on strictly proper scoring rules which improve robustness and accuracy on unseen HF data. Through a set of analytic and engineering examples, we demonstrate that our approach provides a high predictive power while quantifying various sources uncertainties.