# Designing Accurate Emulators for Scientific Processes using Calibration-Driven Deep Models

arXiv: https://arxiv.org/abs/2005.02328 · HTML: https://mlsanity.com/paper/2005.02328/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2020.

## Built on by (most cited first, 8 of 8)

- [A Large-Scale Study of Probabilistic Calibration in Neural Network Regression](https://mlsanity.com/paper/2306.02738/) (2023)
- [Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers](https://mlsanity.com/paper/2103.09360/) (2021)
- [A Spatio-Temporal Neural Network Forecasting Approach for Emulation of Firefront Models](https://mlsanity.com/paper/2206.08523/) (2022)
- [h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective](https://mlsanity.com/paper/2506.17968/) (2025)
- [Self-learning Emulators and Eigenvector Continuation](https://mlsanity.com/paper/2107.13449/) (2021)
- [DBCal: Density Based Calibration of classifier predictions for uncertainty quantification](https://mlsanity.com/paper/2204.00150/) (2022)
- [Explainable Artificial Intelligence in Biomedical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2507.07148/) (2025)
- [An Emulation Framework for Fire Front Spread](https://mlsanity.com/paper/2203.12160/) (2022)

## Abstract (arXiv)

Predictive models that accurately emulate complex scientific processes can achieve exponential speed-ups over numerical simulators or experiments, and at the same time provide surrogates for improving the subsequent analysis. Consequently, there is a recent surge in utilizing modern machine learning (ML) methods, such as deep neural networks, to build data-driven emulators. While the majority of existing efforts has focused on tailoring off-the-shelf ML solutions to better suit the scientific problem at hand, we study an often overlooked, yet important, problem of choosing loss functions to measure the discrepancy between observed data and the predictions from a model. Due to lack of better priors on the expected residual structure, in practice, simple choices such as the mean squared error and the mean absolute error are made. However, the inherent symmetric noise assumption made by these loss functions makes them inappropriate in cases where the data is heterogeneous or when the noise distribution is asymmetric. We propose Learn-by-Calibrating (LbC), a novel deep learning approach based on interval calibration for designing emulators in scientific applications, that are effective even with heterogeneous data and are robust to outliers. Using a large suite of use-cases, we show that LbC provides significant improvements in generalization error over widely-adopted loss function choices, achieves high-quality emulators even in small data regimes and more importantly, recovers the inherent noise structure without any explicit priors.