# DeepUQ: Assessing the Aleatoric Uncertainties from two Deep Learning Methods

arXiv: https://arxiv.org/abs/2411.08587 · HTML: https://mlsanity.com/paper/2411.08587/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 28 references that are in the map.
- First posted on arXiv 13 Nov 2024.

## Builds on (most cited first, 15 of 15)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks](https://mlsanity.com/paper/2203.09168/) (2022)
- [Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation](https://mlsanity.com/paper/2110.03051/) (2021)
- [Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification](https://mlsanity.com/paper/2109.10254/) (2021)
- [The Unreasonable Effectiveness of Deep Evidential Regression](https://mlsanity.com/paper/2205.10060/) (2022)
- [Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithms](https://mlsanity.com/paper/2004.10710/) (2020)
- [AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification](https://mlsanity.com/paper/2110.13511/) (2021)
- [Unified Uncertainties: Combining Input, Data and Model Uncertainty into a Single Formulation](https://mlsanity.com/paper/2406.18787/) (2024)
- [Evaluating Scalable Uncertainty Estimation Methods for DNN-Based Molecular Property Prediction](https://mlsanity.com/paper/1910.03127/) (2019)

## Built on by (most cited first, 2 of 2)

- [On Uncertainty Calibration for Equivariant Functions](https://mlsanity.com/paper/2510.21691/) (2025)
- [A formal definition and meta-model for a machine theory of mind](https://mlsanity.com/paper/2606.03471/) (2026)

## Abstract (arXiv)

Assessing the quality of aleatoric uncertainty estimates from uncertainty quantification (UQ) deep learning methods is important in scientific contexts, where uncertainty is physically meaningful and important to characterize and interpret exactly. We systematically compare aleatoric uncertainty measured by two UQ techniques, Deep Ensembles (DE) and Deep Evidential Regression (DER). Our method focuses on both zero-dimensional (0D) and two-dimensional (2D) data, to explore how the UQ methods function for different data dimensionalities. We investigate uncertainty injected on the input and output variables and include a method to propagate uncertainty in the case of input uncertainty so that we can compare the predicted aleatoric uncertainty to the known values. We experiment with three levels of noise. The aleatoric uncertainty predicted across all models and experiments scales with the injected noise level. However, the predicted uncertainty is miscalibrated to rmstd(σrm al) with the true uncertainty for half of the DE experiments and almost all of the DER experiments. The predicted uncertainty is the least accurate for both UQ methods for the 2D input uncertainty experiment and the high-noise level. While these results do not apply to more complex data, they highlight that further research on post-facto calibration for these methods would be beneficial, particularly for high-noise and high-dimensional settings.