# A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning

arXiv: https://arxiv.org/abs/1801.05075 · HTML: https://mlsanity.com/paper/1801.05075/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2018.

## Built on by (most cited first, 21 of 21)

- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [Framework for Evaluating Faithfulness of Local Explanations](https://mlsanity.com/paper/2202.00734/) (2022)
- [Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2305.06360/) (2023)
- [Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction](https://mlsanity.com/paper/2409.00265/) (2024)
- [Explainable AI for clinical and remote health applications: a survey on tabular and time series data](https://mlsanity.com/paper/2209.06528/) (2022)
- [Towards Explainable Evaluation Metrics for Natural Language Generation](https://mlsanity.com/paper/2203.11131/) (2022)
- [A Fine-grained Interpretability Evaluation Benchmark for Neural NLP](https://mlsanity.com/paper/2205.11097/) (2022)
- [Improving Attribution Methods by Learning Submodular Functions](https://mlsanity.com/paper/2104.09073/) (2021)
- [On Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI System](https://mlsanity.com/paper/2112.01016/) (2021)
- [Pitfalls of Explainable ML: An Industry Perspective](https://mlsanity.com/paper/2106.07758/) (2021)
- [An Interpretability Evaluation Benchmark for Pre-trained Language Models](https://mlsanity.com/paper/2207.13948/) (2022)
- [Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis](https://mlsanity.com/paper/2309.11987/) (2023)
- [SIDU-TXT: An XAI Algorithm for NLP with a Holistic Assessment Approach](https://mlsanity.com/paper/2402.03043/) (2024)
- [Interpretation Quality Score for Measuring the Quality of interpretability methods](https://mlsanity.com/paper/2205.12254/) (2022)
- [Explainable Goal Recognition: A Framework Based on Weight of Evidence](https://mlsanity.com/paper/2303.05622/) (2023)
- [Explaining Natural Language Processing Classifiers with Occlusion and Language Modeling](https://mlsanity.com/paper/2101.11889/) (2021)
- [Towards Human-Centred Explainability Benchmarks For Text Classification](https://mlsanity.com/paper/2211.05452/) (2022)
- [Not All Explanations are Created Equal: Investigating the Pitfalls of Current XAI Evaluation](https://mlsanity.com/paper/2511.03730/) (2025)
- [VitrAI -- Applying Explainable AI in the Real World](https://mlsanity.com/paper/2102.06518/) (2021)
- [Towards Explainable Goal Recognition Using Weight of Evidence (WoE): A Human-Centered Approach](https://mlsanity.com/paper/2409.11675/) (2024)

## Abstract (arXiv)

Research in interpretable machine learning proposes different computational and human subject approaches to evaluate model saliency explanations. These approaches measure different qualities of explanations to achieve diverse goals in designing interpretable machine learning systems. In this paper, we propose a human attention benchmark for image and text domains using multi-layer human attention masks aggregated from multiple human annotators. We then present an evaluation study to evaluate model saliency explanations obtained using Grad-cam and LIME techniques. We demonstrate our benchmark's utility for quantitative evaluation of model explanations by comparing it with human subjective ratings and ground-truth single-layer segmentation masks evaluations. Our study results show that our threshold agnostic evaluation method with the human attention baseline is more effective than single-layer object segmentation masks to ground truth. Our experiments also reveal user biases in the subjective rating of model saliency explanations.