# Evaluating Explanation Without Ground Truth in Interpretable Machine Learning

arXiv: https://arxiv.org/abs/1907.06831 · HTML: https://mlsanity.com/paper/1907.06831/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2019.

## Built on by (most cited first, 29 of 29)

- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing](https://mlsanity.com/paper/2102.12060/) (2021)
- [Tracr: Compiled Transformers as a Laboratory for Interpretability](https://mlsanity.com/paper/2301.05062/) (2023)
- [Few-Shot Self-Rationalization with Natural Language Prompts](https://mlsanity.com/paper/2111.08284/) (2021)
- [A Review on Explainability in Multimodal Deep Neural Nets](https://mlsanity.com/paper/2105.07878/) (2021)
- [Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era](https://mlsanity.com/paper/2403.08946/) (2024)
- [Robust Feature-Level Adversaries are Interpretability Tools](https://mlsanity.com/paper/2110.03605/) (2021)
- [A Fine-grained Interpretability Evaluation Benchmark for Neural NLP](https://mlsanity.com/paper/2205.11097/) (2022)
- [Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absent](https://mlsanity.com/paper/2502.11267/) (2025)
- [On Benchmarking Code LLMs for Android Malware Analysis](https://mlsanity.com/paper/2504.00694/) (2025)
- [Developing a Fidelity Evaluation Approach for Interpretable Machine Learning](https://mlsanity.com/paper/2106.08492/) (2021)
- [Explaining Classifiers by Constructing Familiar Concepts](https://mlsanity.com/paper/2203.04109/) (2022)
- [Interactive Machine Learning: A State of the Art Review](https://mlsanity.com/paper/2207.06196/) (2022)
- [May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainability](https://mlsanity.com/paper/2309.13965/) (2023)
- [Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting](https://mlsanity.com/paper/2410.12284/) (2024)
- [Detection Accuracy for Evaluating Compositional Explanations of Units](https://mlsanity.com/paper/2109.07804/) (2021)
- [Development and Application of Self-Supervised Machine Learning for Smoke Plume and Active Fire Identification from the FIREX-AQ Datasets](https://mlsanity.com/paper/2501.15343/) (2025)
- [Attribution Explanations for Deep Neural Networks: A Theoretical Perspective](https://mlsanity.com/paper/2508.07636/) (2025)
- [LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactuals](https://mlsanity.com/paper/2601.10700/) (2026)
- [In Defence of Post-hoc Explainability](https://mlsanity.com/paper/2412.17883/) (2024)
- [Conversational Explanations: Discussing Explainable AI with Non-AI Experts](https://mlsanity.com/paper/2503.16444/) (2025)
- [When a CBR in Hand is Better than Twins in the Bush](https://mlsanity.com/paper/2305.05111/) (2023)
- [Explainable AI (XAI) for Arrhythmia detection from electrocardiograms](https://mlsanity.com/paper/2508.17294/) (2025)
- [Towards a Transparent and Interpretable AI Model for Medical Image Classifications](https://mlsanity.com/paper/2509.16685/) (2025)
- [Fusing Multi- and Hyperspectral Satellite Data for Harmful Algal Bloom Monitoring with Self-Supervised and Hierarchical Deep Learning](https://mlsanity.com/paper/2510.02763/) (2025)
- [Harnessing Self-Supervised Deep Learning and Geostationary Remote Sensing for Advancing Wildfire and Associated Air Quality Monitoring: Improved Smoke and Fire Front Masking using GOES and TEMPO Radiance Data](https://mlsanity.com/paper/2510.09845/) (2025)
- [A Rectification-Based Approach for Distilling Boosted Trees into Decision Trees](https://mlsanity.com/paper/2510.18615/) (2025)
- [Integrating Feature Understandability into Feature Selection for Human-Centred Tabular Explanations](https://mlsanity.com/paper/2610.16627/) (2026)

## Abstract (arXiv)

Interpretable Machine Learning (IML) has become increasingly important in many real-world applications, such as autonomous cars and medical diagnosis, where explanations are significantly preferred to help people better understand how machine learning systems work and further enhance their trust towards systems. However, due to the diversified scenarios and subjective nature of explanations, we rarely have the ground truth for benchmark evaluation in IML on the quality of generated explanations. Having a sense of explanation quality not only matters for assessing system boundaries, but also helps to realize the true benefits to human users in practical settings. To benchmark the evaluation in IML, in this article, we rigorously define the problem of evaluating explanations, and systematically review the existing efforts from state-of-the-arts. Specifically, we summarize three general aspects of explanation (i.e., generalizability, fidelity and persuasibility) with formal definitions, and respectively review the representative methodologies for each of them under different tasks. Further, a unified evaluation framework is designed according to the hierarchical needs from developers and end-users, which could be easily adopted for different scenarios in practice. In the end, open problems are discussed, and several limitations of current evaluation techniques are raised for future explorations.