# How explainable AI affects human performance: A systematic review of the behavioural consequences of saliency maps

arXiv: https://arxiv.org/abs/2404.16042 · HTML: https://mlsanity.com/paper/2404.16042/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 88 references that are in the map.
- First posted on arXiv 3 Apr 2024.

## Builds on (most cited first, 30 of 48)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Multimodal Explanations: Justifying Decisions and Pointing to the Evidence](https://mlsanity.com/paper/1802.08129/) (2018)
- [CRAFT: Concept Recursive Activation FacTorization for Explainability](https://mlsanity.com/paper/2211.10154/) (2022)
- [Debugging Tests for Model Explanations](https://mlsanity.com/paper/2011.05429/) (2020)
- [From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation](https://mlsanity.com/paper/2206.03208/) (2022)
- [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020)
- [Reliable Post hoc Explanations: Modeling Uncertainty in Explainability](https://mlsanity.com/paper/2008.05030/) (2020)
- [XRAI: Better Attributions Through Regions](https://mlsanity.com/paper/1906.02825/) (2019)
- [HIVE: Evaluating the Human Interpretability of Visual Explanations](https://mlsanity.com/paper/2112.03184/) (2021)
- [What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods](https://mlsanity.com/paper/2112.04417/) (2021)
- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- [Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Study](https://mlsanity.com/paper/2002.00772/) (2020)
- [Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations](https://mlsanity.com/paper/2210.11584/) (2022)
- [Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation](https://mlsanity.com/paper/2212.04629/) (2022)
- [Do Explanations make VQA Models more Predictable to a Human?](https://mlsanity.com/paper/1810.12366/) (2018)
- [Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution](https://mlsanity.com/paper/1912.12191/) (2019)
- [In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Making](https://mlsanity.com/paper/2305.07722/) (2023)
- [How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labels](https://mlsanity.com/paper/2008.11721/) (2020)
- [Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction](https://mlsanity.com/paper/2007.12248/) (2020)
- [One Explanation is Not Enough: Structured Attention Graphs for Image Classification](https://mlsanity.com/paper/2011.06733/) (2020)
- [Teaching Categories to Human Learners with Visual Explanations](https://mlsanity.com/paper/1802.06924/) (2018)
- [A Meta-Analysis of the Utility of Explainable Artificial Intelligence in Human-AI Decision-Making](https://mlsanity.com/paper/2205.05126/) (2022)
- [Explainable Artificial Intelligence (XAI) from a user perspective- A synthesis of prior literature and problematizing avenues for future research](https://mlsanity.com/paper/2211.15343/) (2022)
- [A Study on Multimodal and Interactive Explanations for Visual Question Answering](https://mlsanity.com/paper/2003.00431/) (2020)
- [Embedding Deep Networks into Visual Explanations](https://mlsanity.com/paper/1709.05360/) (2017)
- [Explainable Artificial Intelligence for Human Decision-Support System in Medical Domain](https://mlsanity.com/paper/2105.02357/) (2021)
- [ODAM: Gradient-based instance-specific visual explanations for object detection](https://mlsanity.com/paper/2304.06354/) (2023)

## Built on by (most cited first, 6 of 6)

- [On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs](https://mlsanity.com/paper/2407.19200/) (2024)
- [DocVXQA: Context-Aware Visual Explanations for Document Question Answering](https://mlsanity.com/paper/2505.07496/) (2025)
- [Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps](https://mlsanity.com/paper/2504.17023/) (2025)
- [WebXAII: an open-source web framework to study human-XAI interaction](https://mlsanity.com/paper/2506.14777/) (2025)
- [Associations between echocardiographic traits and AI-ECG predictions of heart failure](https://mlsanity.com/paper/2605.24576/) (2026)
- [Federated Explainable Artificial Intelligence: Roles, Architectures, Evaluation, and Open Challenges](https://mlsanity.com/paper/2607.13045/) (2026)

## Abstract (arXiv)

Saliency maps can explain how deep neural networks classify images. But are they actually useful for humans? The present systematic review of 68 user studies found that while saliency maps can enhance human performance, null effects or even costs are quite common. To investigate what modulates these effects, the empirical outcomes were organised along several factors related to the human tasks, AI performance, XAI methods, images to be classified, human participants and comparison conditions. In image-focused tasks, benefits were less common than in AI-focused tasks, but the effects depended on the specific cognitive requirements. Moreover, benefits were usually restricted to incorrect AI predictions in AI-focused tasks but to correct ones in image-focused tasks. XAI-related factors had surprisingly little impact. The evidence was limited for image- and human-related factors and the effects were highly dependent on the comparison conditions. These findings may support the design of future user studies.