# Towards Reliable Uncertainty Quantification via Deep Ensembles in Multi-output Regression Task

arXiv: https://arxiv.org/abs/2303.16210 · HTML: https://mlsanity.com/paper/2303.16210/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 112 references that are in the map.
- First posted on arXiv 28 Mar 2023.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Deep Exploration via Bootstrapped DQN](https://mlsanity.com/paper/1602.04621/) (2016)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [pymoo: Multi-objective Optimization in Python](https://mlsanity.com/paper/2002.04504/) (2020)
- [Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision](https://mlsanity.com/paper/1906.01620/) (2019)
- [Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling](https://mlsanity.com/paper/1802.09127/) (2018)
- [Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems](https://mlsanity.com/paper/1809.08327/) (2018)
- [Uncertainty Quantification and Deep Ensembles](https://mlsanity.com/paper/2007.08792/) (2020)
- [Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow](https://mlsanity.com/paper/1802.07095/) (2018)
- [Evaluating and Calibrating Uncertainty Prediction in Regression Tasks](https://mlsanity.com/paper/1905.11659/) (2019)
- [Multi-fidelity Bayesian Neural Networks: Algorithms and Applications](https://mlsanity.com/paper/2012.13294/) (2020)
- [Is MC Dropout Bayesian?](https://mlsanity.com/paper/2110.04286/) (2021)
- [AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification](https://mlsanity.com/paper/2110.13511/) (2021)
- [Variational Bayesian dropout: pitfalls and fixes](https://mlsanity.com/paper/1807.01969/) (2018)
- [Variational Inference for Gaussian Process Models with Linear Complexity](https://mlsanity.com/paper/1711.10127/) (2017)
- [Recalibration of Aleatoric and Epistemic Regression Uncertainty in Medical Imaging](https://mlsanity.com/paper/2104.12376/) (2021)
- [Physics-aware Reduced-order Modeling of Transonic Flow via β-Variational Autoencoder](https://mlsanity.com/paper/2205.00608/) (2022)
- [Inverse design optimization framework via a two-step deep learning approach: application to a wind turbine airfoil](https://mlsanity.com/paper/2108.08500/) (2021)
- [Variational Gaussian Dropout is not Bayesian](https://mlsanity.com/paper/1711.02989/) (2017)
- [Should Ensemble Members Be Calibrated?](https://mlsanity.com/paper/2101.05397/) (2021)
- [Calibrating Uncertainties in Object Localization Task](https://mlsanity.com/paper/1811.11210/) (2018)
- [Learning to Predict Error for MRI Reconstruction](https://mlsanity.com/paper/2002.05582/) (2020)

## Built on by (most cited first, 3 of 3)

- [Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspective](https://mlsanity.com/paper/2401.08667/) (2024)
- [Enhancing Graph U-Nets for Mesh-Agnostic Spatio-Temporal Flow Prediction](https://mlsanity.com/paper/2406.03789/) (2024)
- [A Multi-Agent System for Motor Design Optimization via an FEA-AI Hybrid Approach](https://mlsanity.com/paper/2606.09037/) (2026)

## Abstract (arXiv)

This study aims to comprehensively investigate the deep ensemble approach, an approximate Bayesian inference, in the multi-output regression task for predicting the aerodynamic performance of a missile configuration. To this end, the effect of the number of neural networks used in the ensemble, which has been blindly adopted in previous studies, is scrutinized. As a result, an obvious trend towards underestimation of uncertainty as it increases is observed for the first time, and in this context, we propose the deep ensemble framework that applies the post-hoc calibration method to improve its uncertainty quantification performance. It is compared with Gaussian process regression and is shown to have superior performance in terms of regression accuracy (uparrow55~56%), reliability of estimated uncertainty (uparrow38~77%), and training efficiency (uparrow78%). Finally, the potential impact of the suggested framework on the Bayesian optimization is briefly examined, indicating that deep ensemble without calibration may lead to unintended exploratory behavior. This UQ framework can be seamlessly applied and extended to any regression task, as no special assumptions have been made for the specific problem used in this study.