# Explainability through uncertainty: Trustworthy decision-making with neural networks

arXiv: https://arxiv.org/abs/2403.10168 · HTML: https://mlsanity.com/paper/2403.10168/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 51 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 17 of 17)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [The Case for Bayesian Deep Learning](https://mlsanity.com/paper/2001.10995/) (2020)
- [Aleatoric and Epistemic Uncertainty with Random Forests](https://mlsanity.com/paper/2001.00893/) (2020)
- [Deep learning in business analytics and operations research: Models, applications and managerial implications](https://mlsanity.com/paper/1806.10897/) (2018)
- [Performance measures for classification systems with rejection](https://mlsanity.com/paper/1504.02763/) (2015)
- [Forecasting remaining useful life: Interpretable deep learning approach via variational Bayesian inferences](https://mlsanity.com/paper/1907.05146/) (2019)
- [Student Success Prediction in MOOCs](https://mlsanity.com/paper/1711.06349/) (2017)
- [Exploring Bayesian Deep Learning for Urgent Instructor Intervention Need in MOOC Forums](https://mlsanity.com/paper/2104.12643/) (2021)
- [Bayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detection](https://mlsanity.com/paper/2202.12653/) (2022)
- [Learning Uncertainty with Artificial Neural Networks for Improved Predictive Process Monitoring](https://mlsanity.com/paper/2206.06317/) (2022)
- [Predicting student performance using sequence classification with time-based windows](https://mlsanity.com/paper/2208.07749/) (2022)
- [Autoencoders for strategic decision support](https://mlsanity.com/paper/2005.01075/) (2020)

## Built on by (most cited first, 12 of 12)

- [Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing](https://mlsanity.com/paper/2507.10587/) (2025)
- [Decision Information Meets Large Language Models: The Future of Explainable Operations Research](https://mlsanity.com/paper/2502.09994/) (2025)
- [HybridFlow: Quantification of Aleatoric and Epistemic Uncertainty with a Single Hybrid Model](https://mlsanity.com/paper/2510.05054/) (2025)
- [Robust Explanations Through Uncertainty Decomposition: A Path to Trustworthier AI](https://mlsanity.com/paper/2507.12913/) (2025)
- [Concerning Uncertainty -- A Systematic Survey of Uncertainty-Aware XAI](https://mlsanity.com/paper/2603.26838/) (2026)
- [Generalizing Machine Learning Evaluation through the Integration of Shannon Entropy and Rough Set Theory](https://mlsanity.com/paper/2404.12511/) (2024)
- [Active Learning to Guide Labeling Efforts for Question Difficulty Estimation](https://mlsanity.com/paper/2409.09258/) (2024)
- [Ordinality in Discrete-level Question Difficulty Estimation: Introducing Balanced DRPS and OrderedLogitNN](https://mlsanity.com/paper/2507.00736/) (2025)
- [Quantum parameter estimation with uncertainty quantification from continuous measurement data using neural network ensembles](https://mlsanity.com/paper/2509.10756/) (2025)
- [Uncertainty-Guided Expert-AI Collaboration for Efficient Soil Horizon Annotation](https://mlsanity.com/paper/2509.24873/) (2025)
- [How Understanding Forecast Uncertainty Resolves the Explainability Problem in Machine Learning Models](https://mlsanity.com/paper/2602.00179/) (2026)
- [ConformaDecompose: Explaining Uncertainty via Calibration Localization](https://mlsanity.com/paper/2604.27149/) (2026)

## Abstract (arXiv)

Uncertainty is a key feature of any machine learning model and is particularly important in neural networks, which tend to be overconfident. This overconfidence is worrying under distribution shifts, where the model performance silently degrades as the data distribution diverges from the training data distribution. Uncertainty estimation offers a solution to overconfident models, communicating when the output should (not) be trusted. Although methods for uncertainty estimation have been developed, they have not been explicitly linked to the field of explainable artificial intelligence (XAI). Furthermore, literature in operations research ignores the actionability component of uncertainty estimation and does not consider distribution shifts. This work proposes a general uncertainty framework, with contributions being threefold: (i) uncertainty estimation in ML models is positioned as an XAI technique, giving local and model-specific explanations; (ii) classification with rejection is used to reduce misclassifications by bringing a human expert in the loop for uncertain observations; (iii) the framework is applied to a case study on neural networks in educational data mining subject to distribution shifts. Uncertainty as XAI improves the model's trustworthiness in downstream decision-making tasks, giving rise to more actionable and robust machine learning systems in operations research.