Towards Reliable Uncertainty Quantification via Deep Ensembles in Multi-output Regression Task
Not on the map. Why?
Abstract (arXiv)
This study aims to comprehensively investigate the deep ensemble approach, an approximate Bayesian inference, in the multi-output regression task for predicting the aerodynamic performance of a missile configuration. To this end, the effect of the number of neural networks used in the ensemble, which has been blindly adopted in previous studies, is scrutinized. As a result, an obvious trend towards underestimation of uncertainty as it increases is observed for the first time, and in this context, we propose the deep ensemble framework that applies the post-hoc calibration method to improve its uncertainty quantification performance. It is compared with Gaussian process regression and is shown to have superior performance in terms of regression accuracy (uparrow55~56%), reliability of estimated uncertainty (uparrow38~77%), and training efficiency (uparrow78%). Finally, the potential impact of the suggested framework on the Bayesian optimization is briefly examined, indicating that deep ensemble without calibration may lead to unintended exploratory behavior. This UQ framework can be seamlessly applied and extended to any regression task, as no special assumptions have been made for the specific problem used in this study.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 35 of its 112 references that are in the map. First posted on arXiv 28 Mar 2023.
Builds on
35 of 112 references.
References · 35
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
- 2016Deep Exploration via Bootstrapped DQNcited by 518
- 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
- 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
- 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 417
- 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 208
- 2020pymoo: Multi-objective Optimization in Pythoncited by 196
23 more
- 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
- 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
- 2018Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problemscited by 144
- 2020Uncertainty Quantification and Deep Ensemblescited by 138
- 2018Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flowcited by 135
- 2019Evaluating and Calibrating Uncertainty Prediction in Regression Taskscited by 85
- 2020Multi-fidelity Bayesian Neural Networks: Algorithms and Applicationscited by 59
- 2021Is MC Dropout Bayesian?cited by 35
- 2021AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantificationcited by 31
- 2018Variational Bayesian dropout: pitfalls and fixescited by 22
- 2017Variational Inference for Gaussian Process Models with Linear Complexitycited by 21
- 2021Recalibration of Aleatoric and Epistemic Regression Uncertainty in Medical Imagingcited by 18
- 2022Physics-aware Reduced-order Modeling of Transonic Flow via β-Variational Autoencodercited by 15
- 2021Inverse design optimization framework via a two-step deep learning approach: application to a wind turbine airfoilcited by 14
- 2017Variational Gaussian Dropout is not Bayesiancited by 13
- 2021Should Ensemble Members Be Calibrated?cited by 11
- 2018Calibrating Uncertainties in Object Localization Taskcited by 9
- 2020Learning to Predict Error for MRI Reconstructioncited by 9
- 2021Generalisation effects of predictive uncertainty estimation in deep learning for digital pathologycited by 9
- 2021Non-intrusive Surrogate Modeling for Parametrized Time-dependent PDEs using Convolutional Autoencoderscited by 6
- 2021Giving Commands to a Self-Driving Car: How to Deal with Uncertain Situations?cited by 6
- 2022Uncertainty-Aware Mixed-Variable Machine Learning for Materials Designcited by 6
- 2023Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensemblescited by 5
Built on by
Citing papers per year: 2024: 2; 2025: 0; 2026: 1.
3 papers.