# Suppressing simulation bias using multi-modal data

arXiv: https://arxiv.org/abs/2104.09684 · HTML: https://mlsanity.com/paper/2104.09684/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 43 references that are in the map.
- First posted on arXiv 19 Apr 2021.

## Builds on (most cited first, 9 of 9)

- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking](https://mlsanity.com/paper/1912.07748/) (2019)
- [Transfer learning to model inertial confinement fusion experiments](https://mlsanity.com/paper/1812.06055/) (2018)
- [Improved Surrogates in Inertial Confinement Fusion with Manifold and Cycle Consistencies](https://mlsanity.com/paper/1912.08113/) (2019)
- [Cosmological N-body simulations: a challenge for scalable generative models](https://mlsanity.com/paper/1908.05519/) (2019)
- [Cognitive simulation models for inertial confinement fusion: Combining simulation and experimental data](https://mlsanity.com/paper/2103.10590/) (2021)
- [Meaningful uncertainties from deep neural network surrogates of large-scale numerical simulations](https://mlsanity.com/paper/2010.13749/) (2020)

## Built on by (most cited first, 1 of 1)

- [Transformer-Powered Surrogates Close the ICF Simulation-Experiment Gap with Extremely Limited Data](https://mlsanity.com/paper/2312.03642/) (2023)

## Abstract (arXiv)

Many problems in science and engineering require making predictions based on few observations. To build a robust predictive model, these sparse data may need to be augmented with simulated data, especially when the design space is multi-dimensional. Simulations, however, often suffer from an inherent bias. Estimation of this bias may be poorly constrained not only because of data sparsity, but also because traditional predictive models fit only one type of observed outputs, such as scalars or images, instead of all available output data modalities, which might have been acquired and simulated at great cost. To break this limitation and open up the path for multi-modal calibration, we propose to combine a novel, transfer learning technique for suppressing the bias with recent developments in deep learning, which allow building predictive models with multi-modal outputs. First, we train an initial neural network model on simulated data to learn important correlations between different output modalities and between simulation inputs and outputs. Then, the model is partially retrained, or transfer learned, to fit the experiments; a method that has never been implemented in this type of architecture. Using fewer than 10 inertial confinement fusion experiments for training, transfer learning systematically improves the simulation predictions while a simple output calibration, which we design as a baseline, makes the predictions worse. We also offer extensive cross-validation with real and carefully designed synthetic data. The method described in this paper can be applied to a wide range of problems that require transferring knowledge from simulations to the domain of experiments.