# Uncertainty Quantification for Local Model Explanations Without Model Access

arXiv: https://arxiv.org/abs/2301.05761 · HTML: https://mlsanity.com/paper/2301.05761/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 43 references that are in the map.
- First posted on arXiv 13 Jan 2023.

## Builds on (most cited first, 23 of 23)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training](https://mlsanity.com/paper/2106.01342/) (2021)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Reliable Post hoc Explanations: Modeling Uncertainty in Explainability](https://mlsanity.com/paper/2008.05030/) (2020)
- [CXPlain: Causal Explanations for Model Interpretation under Uncertainty](https://mlsanity.com/paper/1910.12336/) (2019)
- [Predictive Inequity in Object Detection](https://mlsanity.com/paper/1902.11097/) (2019)
- [Rethinking Stability for Attribution-based Explanations](https://mlsanity.com/paper/2203.06877/) (2022)
- [Human-Centered Artificial Intelligence and Machine Learning](https://mlsanity.com/paper/1901.11184/) (2019)
- [U-CAM: Visual Explanation using Uncertainty based Class Activation Maps](https://mlsanity.com/paper/1908.06306/) (2019)
- [Probing GNN Explainers: A Rigorous Theoretical and Empirical Analysis of GNN Explanation Methods](https://mlsanity.com/paper/2106.09078/) (2021)
- [How Much Can I Trust You? -- Quantifying Uncertainties in Explaining Neural Networks](https://mlsanity.com/paper/2006.09000/) (2020)
- [Uncertainty-Aware Deep Ensembles for Reliable and Explainable Predictions of Clinical Time Series](https://mlsanity.com/paper/2010.11310/) (2020)
- [Cohort Shapley value for algorithmic fairness](https://mlsanity.com/paper/2105.07168/) (2021)
- [Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach](https://mlsanity.com/paper/2111.09121/) (2021)
- [Model free variable importance for high dimensional data](https://mlsanity.com/paper/2211.08414/) (2022)
- [Bootstrap Inference for Multiple Imputation under Uncongeniality and Misspecification](https://mlsanity.com/paper/1911.09980/) (2019)

## Built on by (most cited first, 1 of 1)

- [Trustworthy AI Must Account for Interactions](https://mlsanity.com/paper/2504.07170/) (2025)

## Abstract (arXiv)

We present a model-agnostic algorithm for generating post-hoc explanations and uncertainty intervals for a machine learning model when only a static sample of inputs and outputs from the model is available, rather than direct access to the model itself. This situation may arise when model evaluations are expensive; when privacy, security and bandwidth constraints are imposed; or when there is a need for real-time, on-device explanations. Our algorithm uses a bootstrapping approach to quantify the uncertainty that inevitably arises when generating explanations from a finite sample of model queries. Through a simulation study, we show that the uncertainty intervals generated by our algorithm exhibit a favorable trade-off between interval width and coverage probability compared to the naive confidence intervals from classical regression analysis as well as current Bayesian approaches for quantifying explanation uncertainty. We further demonstrate the capabilities of our method by applying it to black-box models, including a deep neural network, trained on three real-world datasets.