# Bayesian optimized deep ensemble for uncertainty quantification of deep neural networks: a system safety case study on sodium fast reactor thermal stratification modeling

arXiv: https://arxiv.org/abs/2412.08776 · HTML: https://mlsanity.com/paper/2412.08776/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 68 references that are in the map.
- First posted on arXiv 11 Dec 2024.

## Builds on (most cited first, 11 of 11)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Hyperparameter Ensembles for Robustness and Uncertainty Quantification](https://mlsanity.com/paper/2006.13570/) (2020)
- [BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization](https://mlsanity.com/paper/1910.06403/) (2019)
- [Recalibration of Aleatoric and Epistemic Regression Uncertainty in Medical Imaging](https://mlsanity.com/paper/2104.12376/) (2021)
- [Deep Learning for Bayesian Optimization of Scientific Problems with High-Dimensional Structure](https://mlsanity.com/paper/2104.11667/) (2021)
- [Bayesian Hyperparameter Optimization with BoTorch, GPyTorch and Ax](https://mlsanity.com/paper/1912.05686/) (2019)
- [Predicting Critical Heat Flux with Uncertainty Quantification and Domain Generalization Using Conditional Variational Autoencoders and Deep Neural Networks](https://mlsanity.com/paper/2409.05790/) (2024)
- [NEORL: NeuroEvolution Optimization with Reinforcement Learning](https://mlsanity.com/paper/2112.07057/) (2021)
- [A Safe Reinforcement Learning Algorithm for Supervisory Control of Power Plants](https://mlsanity.com/paper/2401.13020/) (2024)
- [Sparse Bayesian neural networks for regression: Tackling overfitting and computational challenges in uncertainty quantification](https://mlsanity.com/paper/2310.15614/) (2023)

## Built on by (most cited first, 5 of 5)

- [Uncertainty Quantification for Machine Learning in Healthcare: A Survey](https://mlsanity.com/paper/2505.02874/) (2025)
- [Automating Data-Driven Modeling and Analysis for Engineering Applications using Large Language Model Agents](https://mlsanity.com/paper/2510.01398/) (2025)
- [NeuroPareto: Calibrated Acquisition for Costly Many-Goal Search in Vast Parameter Spaces](https://mlsanity.com/paper/2602.03901/) (2026)
- [RADIANT-LLM: an Agentic Retrieval Augmented Generation Framework for Reliable Decision Support in Safety-Critical Nuclear Engineering](https://mlsanity.com/paper/2604.22755/) (2026)
- [Interval and fuzzy physics-augmented neural networks (iPANN and fPANN) for uncertainty quantification and propagation in constitutive modeling](https://mlsanity.com/paper/2607.20339/) (2026)

## Abstract (arXiv)

Accurate predictions and uncertainty quantification (UQ) are essential for decision-making in risk-sensitive fields such as system safety modeling. Deep ensembles (DEs) are efficient and scalable methods for UQ in Deep Neural Networks (DNNs); however, their performance is limited when constructed by simply retraining the same DNN multiple times with randomly sampled initializations. To overcome this limitation, we propose a novel method that combines Bayesian optimization (BO) with DE, referred to as BODE, to enhance both predictive accuracy and UQ. We apply BODE to a case study involving a Densely connected Convolutional Neural Network (DCNN) trained on computational fluid dynamics (CFD) data to predict eddy viscosity in sodium fast reactor thermal stratification modeling. Compared to a manually tuned baseline ensemble, BODE estimates total uncertainty approximately four times lower in a noise-free environment, primarily due to the baseline's overestimation of aleatoric uncertainty. Specifically, BODE estimates aleatoric uncertainty close to zero, while aleatoric uncertainty dominates the total uncertainty in the baseline ensemble. We also observe a reduction of more than 30% in epistemic uncertainty. When Gaussian noise with standard deviations of 5% and 10% is introduced into the data, BODE accurately fits the data and estimates uncertainty that aligns with the data noise. These results demonstrate that BODE effectively reduces uncertainty and enhances predictions in data-driven models, making it a flexible approach for various applications requiring accurate predictions and robust UQ.