# Reliable Post hoc Explanations: Modeling Uncertainty in Explainability

arXiv: https://arxiv.org/abs/2008.05030 · HTML: https://mlsanity.com/paper/2008.05030/

## Key facts

- Cited by 125 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2020.

## Built on by (most cited first, 30 of 125)

- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation](https://mlsanity.com/paper/2306.07304/) (2023)
- [A Survey on Uncertainty Quantification Methods for Deep Learning](https://mlsanity.com/paper/2302.13425/) (2023)
- [TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations](https://mlsanity.com/paper/2207.04154/) (2022)
- [From Shapley Values to Generalized Additive Models and back](https://mlsanity.com/paper/2209.04012/) (2022)
- [Neural Basis Models for Interpretability](https://mlsanity.com/paper/2205.14120/) (2022)
- [Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contexts](https://mlsanity.com/paper/2201.10295/) (2022)
- [Rethinking Explainability as a Dialogue: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01875/) (2022)
- [Rethinking Stability for Attribution-based Explanations](https://mlsanity.com/paper/2203.06877/) (2022)
- [Post Hoc Explanations of Language Models Can Improve Language Models](https://mlsanity.com/paper/2305.11426/) (2023)
- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [Metrics for saliency map evaluation of deep learning explanation methods](https://mlsanity.com/paper/2201.13291/) (2022)
- [Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations](https://mlsanity.com/paper/2205.07277/) (2022)
- [Don't Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysis](https://mlsanity.com/paper/2202.07728/) (2022)
- [Explaining Predictive Uncertainty with Information Theoretic Shapley Values](https://mlsanity.com/paper/2306.05724/) (2023)
- [Guidelines and Evaluation of Clinical Explainable AI in Medical Image Analysis](https://mlsanity.com/paper/2202.10553/) (2022)
- [The Inadequacy of Shapley Values for Explainability](https://mlsanity.com/paper/2302.08160/) (2023)
- [Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimization](https://mlsanity.com/paper/2306.06805/) (2023)
- [Scalable Interpretability via Polynomials](https://mlsanity.com/paper/2205.14108/) (2022)
- [Explainability is NOT a Game](https://mlsanity.com/paper/2307.07514/) (2023)
- [Explaining the Uncertain: Stochastic Shapley Values for Gaussian Process Models](https://mlsanity.com/paper/2305.15167/) (2023)
- [Rational Shapley Values](https://mlsanity.com/paper/2106.10191/) (2021)
- [Distill n' Explain: explaining graph neural networks using simple surrogates](https://mlsanity.com/paper/2303.10139/) (2023)
- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [Calibrating Multimodal Learning](https://mlsanity.com/paper/2306.01265/) (2023)
- [Calibrated Explanations: with Uncertainty Information and Counterfactuals](https://mlsanity.com/paper/2305.02305/) (2023)
- [Efficient XAI Techniques: A Taxonomic Survey](https://mlsanity.com/paper/2302.03225/) (2023)
- [Stability Guarantees for Feature Attributions with Multiplicative Smoothing](https://mlsanity.com/paper/2307.05902/) (2023)
- [Select Wisely and Explain: Active Learning and Probabilistic Local Post-hoc Explainability](https://mlsanity.com/paper/2108.06907/) (2021)

## Abstract (arXiv)

As black box explanations are increasingly being employed to establish model credibility in high-stakes settings, it is important to ensure that these explanations are accurate and reliable. However, prior work demonstrates that explanations generated by state-of-the-art techniques are inconsistent, unstable, and provide very little insight into their correctness and reliability. In addition, these methods are also computationally inefficient, and require significant hyper-parameter tuning. In this paper, we address the aforementioned challenges by developing a novel Bayesian framework for generating local explanations along with their associated uncertainty. We instantiate this framework to obtain Bayesian versions of LIME and KernelSHAP which output credible intervals for the feature importances, capturing the associated uncertainty. The resulting explanations not only enable us to make concrete inferences about their quality (e.g., there is a 95% chance that the feature importance lies within the given range), but are also highly consistent and stable. We carry out a detailed theoretical analysis that leverages the aforementioned uncertainty to estimate how many perturbations to sample, and how to sample for faster convergence. This work makes the first attempt at addressing several critical issues with popular explanation methods in one shot, thereby generating consistent, stable, and reliable explanations with guarantees in a computationally efficient manner. Experimental evaluation with multiple real world datasets and user studies demonstrate that the efficacy of the proposed framework.