# Concerning Uncertainty -- A Systematic Survey of Uncertainty-Aware XAI

arXiv: https://arxiv.org/abs/2603.26838 · HTML: https://mlsanity.com/paper/2603.26838/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 80 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Builds on (most cited first, 27 of 27)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Reliable Post hoc Explanations: Modeling Uncertainty in Explainability](https://mlsanity.com/paper/2008.05030/) (2020)
- [General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models](https://mlsanity.com/paper/2007.04131/) (2020)
- [BayLIME: Bayesian Local Interpretable Model-Agnostic Explanations](https://mlsanity.com/paper/2012.03058/) (2020)
- [Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction](https://mlsanity.com/paper/2409.00265/) (2024)
- [Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties](https://mlsanity.com/paper/2103.08951/) (2021)
- [Modern Monte Carlo Methods for Efficient Uncertainty Quantification and Propagation: A Survey](https://mlsanity.com/paper/2011.00680/) (2020)
- [Tell Me a Story! Narrative-Driven XAI with Large Language Models](https://mlsanity.com/paper/2309.17057/) (2023)
- [Calibrated Explanations: with Uncertainty Information and Counterfactuals](https://mlsanity.com/paper/2305.02305/) (2023)
- [Select Wisely and Explain: Active Learning and Probabilistic Local Post-hoc Explainability](https://mlsanity.com/paper/2108.06907/) (2021)
- [Explainability through uncertainty: Trustworthy decision-making with neural networks](https://mlsanity.com/paper/2403.10168/) (2024)
- [ProtoASNet: Dynamic Prototypes for Inherently Interpretable and Uncertainty-Aware Aortic Stenosis Classification in Echocardiography](https://mlsanity.com/paper/2307.14433/) (2023)
- [One step closer to unbiased aleatoric uncertainty estimation](https://mlsanity.com/paper/2312.10469/) (2023)
- [Towards Modeling Uncertainties of Self-explaining Neural Networks via Conformal Prediction](https://mlsanity.com/paper/2401.01549/) (2024)
- [Explaining Predictive Uncertainty by Exposing Second-Order Effects](https://mlsanity.com/paper/2401.17441/) (2024)
- [Boundary-Aware Uncertainty for Feature Attribution Explainers](https://mlsanity.com/paper/2210.02419/) (2022)
- [Generative Perturbation Analysis for Probabilistic Black-Box Anomaly Attribution](https://mlsanity.com/paper/2308.04708/) (2023)
- [Calibrated Explanations for Regression](https://mlsanity.com/paper/2308.16245/) (2023)
- [Improving Explainable Object-induced Model through Uncertainty for Automated Vehicles](https://mlsanity.com/paper/2402.15572/) (2024)
- [DUE: Dynamic Uncertainty-Aware Explanation Supervision via 3D Imputation](https://mlsanity.com/paper/2403.10831/) (2024)
- [Investigating the Impact of Model Instability on Explanations and Uncertainty](https://mlsanity.com/paper/2402.13006/) (2024)
- [Communicating Uncertainty in Machine Learning Explanations: A Visualization Analytics Approach for Predictive Process Monitoring](https://mlsanity.com/paper/2304.05736/) (2023)
- [ScatterUQ: Interactive Uncertainty Visualizations for Multiclass Deep Learning Problems](https://mlsanity.com/paper/2308.04588/) (2023)

## Built on by (most cited first, 2 of 2)

- [ConformaDecompose: Explaining Uncertainty via Calibration Localization](https://mlsanity.com/paper/2604.27149/) (2026)
- [Beyond Explaining Predictions: Logic-Based Explanations for Confidence in Machine Learning Models](https://mlsanity.com/paper/2606.10347/) (2026)

## Abstract (arXiv)

This paper surveys uncertainty-aware explainable artificial intelligence (UAXAI), examining how uncertainty is incorporated into explanatory pipelines and how such methods are evaluated. Across the literature, three recurring approaches to uncertainty quantification emerge (Bayesian, Monte Carlo, and Conformal methods), alongside distinct strategies for integrating uncertainty into explanations: assessing trustworthiness, constraining models or explanations, and explicitly communicating uncertainty. Evaluation practices remain fragmented and largely model centered, with limited attention to users and inconsistent reporting of reliability properties (e.g., calibration, coverage, explanation stability). Recent work leans towards calibration, distribution free techniques and recognizes explainer variability as a central concern. We argue that progress in UAXAI requires unified evaluation principles that link uncertainty propagation, robustness, and human decision-making, and highlight counterfactual and calibration approaches as promising avenues for aligning interpretability with reliability.