# Explanations are a Means to an End: Decision Theoretic Explanation Evaluation

arXiv: https://arxiv.org/abs/2506.22740 · HTML: https://mlsanity.com/paper/2506.22740/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 68 references that are in the map.
- First posted on arXiv 28 Jun 2025.

## Builds on (most cited first, 30 of 40)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability](https://mlsanity.com/paper/2301.04709/) (2023)
- [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019)
- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- [On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection](https://mlsanity.com/paper/1811.07901/) (2018)
- [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020)
- [When combinations of humans and AI are useful: A systematic review and meta-analysis](https://mlsanity.com/paper/2405.06087/) (2024)
- [Auditing language models for hidden objectives](https://mlsanity.com/paper/2503.10965/) (2025)
- [Explaining the Explainer: A First Theoretical Analysis of LIME](https://mlsanity.com/paper/2001.03447/) (2020)
- [Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences](https://mlsanity.com/paper/1712.00547/) (2017)
- [Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations](https://mlsanity.com/paper/2301.07255/) (2023)
- [MIB: A Mechanistic Interpretability Benchmark](https://mlsanity.com/paper/2504.13151/) (2025)
- [The Pragmatic Turn in Explainable Artificial Intelligence (XAI)](https://mlsanity.com/paper/2002.09595/) (2020)
- [In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Making](https://mlsanity.com/paper/2305.07722/) (2023)
- [U-Calibration: Forecasting for an Unknown Agent](https://mlsanity.com/paper/2307.00168/) (2023)
- [Optimization of Scoring Rules](https://mlsanity.com/paper/2007.02905/) (2020)
- [A Decision Theoretic Framework for Measuring AI Reliance](https://mlsanity.com/paper/2401.15356/) (2024)
- [Logic for Explainable AI](https://mlsanity.com/paper/2305.05172/) (2023)
- [Machine Explanations and Human Understanding](https://mlsanity.com/paper/2202.04092/) (2022)

## Built on by (most cited first, 1 of 1)

- [ComplLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-making](https://mlsanity.com/paper/2602.19458/) (2026)

## Abstract (arXiv)

Explanations of model behavior are commonly evaluated via proxy properties weakly tied to the purposes explanations serve in practice. We contribute a decision theoretic framework that treats explanations as information signals valued by the expected improvement they enable on a specified decision task. This approach yields three distinct estimands: 1) a theoretical benchmark that upperbounds achievable performance by any agent with the explanation, 2) a human-complementary value that quantifies the theoretically attainable value that is not already captured by a baseline human decision policy, and 3) a behavioral value representing the causal effect of providing the explanation to human decision-makers. We instantiate these definitions in a practical validation workflow, and apply them to assess explanation potential and interpret behavioral effects in human-AI decision support and mechanistic interpretability.