# AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification

arXiv: https://arxiv.org/abs/2110.13511 · HTML: https://mlsanity.com/paper/2110.13511/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 16 of 16)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [Hyperparameter Ensembles for Robustness and Uncertainty Quantification](https://mlsanity.com/paper/2006.13570/) (2020)
- [Learning in Implicit Generative Models](https://mlsanity.com/paper/1610.03483/) (2016)
- [High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach](https://mlsanity.com/paper/1802.07167/) (2018)
- [On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty](https://mlsanity.com/paper/2102.11409/) (2021)
- [Neural Ensemble Search for Uncertainty Estimation and Dataset Shift](https://mlsanity.com/paper/2006.08573/) (2020)
- [Multivariate Uncertainty in Deep Learning](https://mlsanity.com/paper/1910.14215/) (2019)
- [AgEBO-Tabular: Joint Neural Architecture and Hyperparameter Search with Autotuned Data-Parallel Training for Tabular Data](https://mlsanity.com/paper/2010.16358/) (2020)

## Built on by (most cited first, 30 of 31)

- [Credal Bayesian Deep Learning](https://mlsanity.com/paper/2302.09656/) (2023)
- [Optimal Training of Mean Variance Estimation Neural Networks](https://mlsanity.com/paper/2302.08875/) (2023)
- [Revisiting Essential and Nonessential Settings of Evidential Deep Learning](https://mlsanity.com/paper/2410.00393/) (2024)
- [Optimizing Distributed Training on Frontier for Large Language Models](https://mlsanity.com/paper/2312.12705/) (2023)
- [Deep-Ensemble-Based Uncertainty Quantification in Spatiotemporal Graph Neural Networks for Traffic Forecasting](https://mlsanity.com/paper/2204.01618/) (2022)
- [A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning](https://mlsanity.com/paper/2406.00332/) (2024)
- [A Benchmark on Uncertainty Quantification for Deep Learning Prognostics](https://mlsanity.com/paper/2302.04730/) (2023)
- [Interpretable Uncertainty Quantification in AI for HEP](https://mlsanity.com/paper/2208.03284/) (2022)
- [Learning Terrain-Aware Kinodynamic Model for Autonomous Off-Road Rally Driving With Model Predictive Path Integral Control](https://mlsanity.com/paper/2305.00676/) (2023)
- [Uncertainty Quantification for Molecular Property Predictions with Graph Neural Architecture Search](https://mlsanity.com/paper/2307.10438/) (2023)
- [Scalable Training of Trustworthy and Energy-Efficient Predictive Graph Foundation Models for Atomistic Materials Modeling: A Case Study with HydraGNN](https://mlsanity.com/paper/2406.12909/) (2024)
- [Credal and Interval Deep Evidential Classifications](https://mlsanity.com/paper/2512.05526/) (2025)
- [Asynchronous Decentralized Bayesian Optimization for Large Scale Hyperparameter Optimization](https://mlsanity.com/paper/2207.00479/) (2022)
- [Sequential Bayesian Neural Subnetwork Ensembles](https://mlsanity.com/paper/2206.00794/) (2022)
- [Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles](https://mlsanity.com/paper/2302.09748/) (2023)
- [Adaptive learning of effective dynamics: Adaptive real-time, online modeling for complex systems](https://mlsanity.com/paper/2304.01732/) (2023)
- [LTAU-FF: Loss Trajectory Analysis for Uncertainty in Atomistic Force Fields](https://mlsanity.com/paper/2402.00853/) (2024)
- [Towards Reliable Uncertainty Quantification via Deep Ensembles in Multi-output Regression Task](https://mlsanity.com/paper/2303.16210/) (2023)
- [Hyp-UML: Hyperbolic Image Retrieval with Uncertainty-aware Metric Learning](https://mlsanity.com/paper/2310.08390/) (2023)
- [Employing Artificial Intelligence to Steer Exascale Workflows with Colmena](https://mlsanity.com/paper/2408.14434/) (2024)
- [Automated Real-time Assessment of Intracranial Hemorrhage Detection AI Using an Ensembled Monitoring Model (EMM)](https://mlsanity.com/paper/2505.11738/) (2025)
- [Deep Evidential Learning for Radiotherapy Dose Prediction](https://mlsanity.com/paper/2404.17126/) (2024)
- [An Uncertainty-aware Deep Learning Framework-based Robust Design Optimization of Metamaterial Units](https://mlsanity.com/paper/2407.20251/) (2024)
- [Neural Network Prediction of Strong Lensing Systems with Domain Adaptation and Uncertainty Quantification](https://mlsanity.com/paper/2411.03334/) (2024)
- [DeepUQ: Assessing the Aleatoric Uncertainties from two Deep Learning Methods](https://mlsanity.com/paper/2411.08587/) (2024)
- [Unified Probabilistic Neural Architecture and Weight Ensembling Improves Model Robustness](https://mlsanity.com/paper/2210.04083/) (2022)
- [Toward Robust Uncertainty Estimation with Random Activation Functions](https://mlsanity.com/paper/2302.14552/) (2023)
- [A Neural Emulator for Uncertainty Estimation of Fire Propagation](https://mlsanity.com/paper/2305.06139/) (2023)
- [Uncertainty Quantification in Graph Neural Networks with Shallow Ensembles](https://mlsanity.com/paper/2504.12627/) (2025)
- [Instance-wise Uncertainty for Class Imbalance in Semantic Segmentation](https://mlsanity.com/paper/2407.12609/) (2024)

## Abstract (arXiv)

Deep neural networks are powerful predictors for a variety of tasks. However, they do not capture uncertainty directly. Using neural network ensembles to quantify uncertainty is competitive with approaches based on Bayesian neural networks while benefiting from better computational scalability. However, building ensembles of neural networks is a challenging task because, in addition to choosing the right neural architecture or hyperparameters for each member of the ensemble, there is an added cost of training each model. We propose AutoDEUQ, an automated approach for generating an ensemble of deep neural networks. Our approach leverages joint neural architecture and hyperparameter search to generate ensembles. We use the law of total variance to decompose the predictive variance of deep ensembles into aleatoric (data) and epistemic (model) uncertainties. We show that AutoDEUQ outperforms probabilistic backpropagation, Monte Carlo dropout, deep ensemble, distribution-free ensembles, and hyper ensemble methods on a number of regression benchmarks.