# Using AI Uncertainty Quantification to Improve Human Decision-Making

arXiv: https://arxiv.org/abs/2309.10852 · HTML: https://mlsanity.com/paper/2309.10852/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 19 Sep 2023.

## Builds on (most cited first, 25 of 25)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks](https://mlsanity.com/paper/1807.07356/) (2018)
- [To Trust Or Not To Trust A Classifier](https://mlsanity.com/paper/1805.11783/) (2018)
- [Multiplicative Normalizing Flows for Variational Bayesian Neural Networks](https://mlsanity.com/paper/1703.01961/) (2017)
- [From Variational to Deterministic Autoencoders](https://mlsanity.com/paper/1903.12436/) (2019)
- [Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty](https://mlsanity.com/paper/2011.07586/) (2020)
- [DEUP: Direct Epistemic Uncertainty Prediction](https://mlsanity.com/paper/2102.08501/) (2021)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play](https://mlsanity.com/paper/1810.09648/) (2018)
- [Does Explainable Artificial Intelligence Improve Human Decision-Making?](https://mlsanity.com/paper/2006.11194/) (2020)
- [Accurate Uncertainty Estimation and Decomposition in Ensemble Learning](https://mlsanity.com/paper/1911.04061/) (2019)
- [On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators](https://mlsanity.com/paper/1610.06665/) (2016)
- [Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles](https://mlsanity.com/paper/1906.07380/) (2019)
- [A Meta-Analysis of the Utility of Explainable Artificial Intelligence in Human-AI Decision-Making](https://mlsanity.com/paper/2205.05126/) (2022)
- [A Human-Grounded Evaluation of SHAP for Alert Processing](https://mlsanity.com/paper/1907.03324/) (2019)
- [UncertaINR: Uncertainty Quantification of End-to-End Implicit Neural Representations for Computed Tomography](https://mlsanity.com/paper/2202.10847/) (2022)
- [Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping](https://mlsanity.com/paper/2206.03633/) (2022)
- [UQGAN: A Unified Model for Uncertainty Quantification of Deep Classifiers trained via Conditional GANs](https://mlsanity.com/paper/2201.13279/) (2022)
- [Liberty or Depth: Deep Bayesian Neural Nets Do Not Need Complex Weight Posterior Approximations](https://mlsanity.com/paper/2002.03704/) (2020)

## Built on by (most cited first, 11 of 11)

- [From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered](https://mlsanity.com/paper/2506.07461/) (2025)
- [Online Algorithms with Uncertainty-Quantified Predictions](https://mlsanity.com/paper/2310.11558/) (2023)
- [Human-AI Collaborative Uncertainty Quantification](https://mlsanity.com/paper/2510.23476/) (2025)
- [HybridFlow: Quantification of Aleatoric and Epistemic Uncertainty with a Single Hybrid Model](https://mlsanity.com/paper/2510.05054/) (2025)
- [Exploring the Impact of Explainable AI and Cognitive Capabilities on Users' Decisions](https://mlsanity.com/paper/2505.01192/) (2025)
- [DeCoDe: Defer-and-Complement Decision-Making via Decoupled Concept Bottleneck Models](https://mlsanity.com/paper/2505.19220/) (2025)
- [Parameter Space Analysis through Guided Visual Interpolations](https://mlsanity.com/paper/2509.19202/) (2025)
- [Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revision](https://mlsanity.com/paper/2604.06723/) (2026)
- [Human Decision-Making with Persuasive and Narrative LLM Explanations](https://mlsanity.com/paper/2605.23867/) (2026)
- [Localizing Input Uncertainty Quantification for Large Language Models via Shapley Values](https://mlsanity.com/paper/2605.28170/) (2026)
- [Explainable Uncertainty Estimation for Reliable Medical AI](https://mlsanity.com/paper/2608.28052/) (2026)

## Abstract (arXiv)

AI Uncertainty Quantification (UQ) has the potential to improve human decision-making beyond AI predictions alone by providing additional probabilistic information to users. The majority of past research on AI and human decision-making has concentrated on model explainability and interpretability, with little focus on understanding the potential impact of UQ on human decision-making. We evaluated the impact on human decision-making for instance-level UQ, calibrated using a strict scoring rule, in two online behavioral experiments. In the first experiment, our results showed that UQ was beneficial for decision-making performance compared to only AI predictions. In the second experiment, we found UQ had generalizable benefits for decision-making across a variety of representations for probabilistic information. These results indicate that implementing high quality, instance-level UQ for AI may improve decision-making with real systems compared to AI predictions alone.