# How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labels

arXiv: https://arxiv.org/abs/2008.11721 · HTML: https://mlsanity.com/paper/2008.11721/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2020.

## Built on by (most cited first, 30 of 39)

- [CRAFT: Concept Recursive Activation FacTorization for Explainability](https://mlsanity.com/paper/2211.10154/) (2022)
- [A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation](https://mlsanity.com/paper/2306.07304/) (2023)
- [HIVE: Evaluating the Human Interpretability of Visual Explanations](https://mlsanity.com/paper/2112.03184/) (2021)
- [What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods](https://mlsanity.com/paper/2112.04417/) (2021)
- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- [Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations](https://mlsanity.com/paper/2210.11584/) (2022)
- ["Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction](https://mlsanity.com/paper/2210.03735/) (2022)
- [Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation](https://mlsanity.com/paper/2212.04629/) (2022)
- [FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methods](https://mlsanity.com/paper/2308.06248/) (2023)
- [Visual correspondence-based explanations improve AI robustness and human-AI team accuracy](https://mlsanity.com/paper/2208.00780/) (2022)
- [ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writing](https://mlsanity.com/paper/2305.09770/) (2023)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [COCKATIEL: COntinuous Concept ranKed ATtribution with Interpretable ELements for explaining neural net classifiers on NLP tasks](https://mlsanity.com/paper/2305.06754/) (2023)
- [Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absent](https://mlsanity.com/paper/2502.11267/) (2025)
- [Do Users Benefit From Interpretable Vision? A User Study, Baseline, And Dataset](https://mlsanity.com/paper/2204.11642/) (2022)
- [FINER: Enhancing State-of-the-art Classifiers with Feature Attribution to Facilitate Security Analysis](https://mlsanity.com/paper/2308.05362/) (2023)
- [Interpreting and Correcting Medical Image Classification with PIP-Net](https://mlsanity.com/paper/2307.10404/) (2023)
- [Human-Centric Research for NLP: Towards a Definition and Guiding Questions](https://mlsanity.com/paper/2207.04447/) (2022)
- [Why is plausibility surprisingly problematic as an XAI criterion?](https://mlsanity.com/paper/2303.17707/) (2023)
- [FACE: Faithful Automatic Concept Extraction](https://mlsanity.com/paper/2510.11675/) (2025)
- [Are Shortest Rationales the Best Explanations for Human Understanding?](https://mlsanity.com/paper/2203.08788/) (2022)
- [Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attribution](https://mlsanity.com/paper/2602.14869/) (2026)
- [How explainable AI affects human performance: A systematic review of the behavioural consequences of saliency maps](https://mlsanity.com/paper/2404.16042/) (2024)
- [Feature Accentuation: Revealing 'What' Features Respond to in Natural Images](https://mlsanity.com/paper/2402.10039/) (2024)
- [Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting](https://mlsanity.com/paper/2410.12284/) (2024)
- [Invisible Users: Uncovering End-Users' Requirements for Explainable AI via Explanation Forms and Goals](https://mlsanity.com/paper/2302.06609/) (2023)
- [Explaining the Road Not Taken](https://mlsanity.com/paper/2103.14973/) (2021)
- [Responsibility: An Example-based Explainable AI approach via Training Process Inspection](https://mlsanity.com/paper/2209.03433/) (2022)
- [From Explanation to Action: An End-to-End Human-in-the-loop Framework for Anomaly Reasoning and Management](https://mlsanity.com/paper/2304.03368/) (2023)
- [Two4Two: Evaluating Interpretable Machine Learning - A Synthetic Dataset For Controlled Experiments](https://mlsanity.com/paper/2105.02825/) (2021)

## Abstract (arXiv)

Explaining to users why automated systems make certain mistakes is important and challenging. Researchers have proposed ways to automatically produce interpretations for deep neural network models. However, it is unclear how useful these interpretations are in helping users figure out why they are getting an error. If an interpretation effectively explains to users how the underlying deep neural network model works, people who were presented with the interpretation should be better at predicting the model's outputs than those who were not. This paper presents an investigation on whether or not showing machine-generated visual interpretations helps users understand the incorrectly predicted labels produced by image classifiers. We showed the images and the correct labels to 150 online crowd workers and asked them to select the incorrectly predicted labels with or without showing them the machine-generated visual interpretations. The results demonstrated that displaying the visual interpretations did not increase, but rather decreased, the average guessing accuracy by roughly 10%.