# A Meta-Analysis of the Utility of Explainable Artificial Intelligence in Human-AI Decision-Making

arXiv: https://arxiv.org/abs/2205.05126 · HTML: https://mlsanity.com/paper/2205.05126/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 70 references that are in the map.
- First posted on arXiv 10 May 2022.

## Builds on (most cited first, 29 of 29)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey](https://mlsanity.com/paper/2006.11371/) (2020)
- [Ex Machina: Personal Attacks Seen at Scale](https://mlsanity.com/paper/1610.08914/) (2016)
- [Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?](https://mlsanity.com/paper/2005.01831/) (2020)
- [On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection](https://mlsanity.com/paper/1811.07901/) (2018)
- [Debugging Tests for Model Explanations](https://mlsanity.com/paper/2011.05429/) (2020)
- [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020)
- [Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening](https://mlsanity.com/paper/1903.08297/) (2019)
- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- ["Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans](https://mlsanity.com/paper/2001.05871/) (2020)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making](https://mlsanity.com/paper/2101.05303/) (2021)
- [Do Explanations make VQA Models more Predictable to a Human?](https://mlsanity.com/paper/1810.12366/) (2018)
- [Does Explainable Artificial Intelligence Improve Human Decision-Making?](https://mlsanity.com/paper/2006.11194/) (2020)
- [Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction](https://mlsanity.com/paper/2007.12248/) (2020)
- [On the Influence of Explainable AI on Automation Bias](https://mlsanity.com/paper/2204.08859/) (2022)
- [On the Effect of Information Asymmetry in Human-AI Teams](https://mlsanity.com/paper/2205.01467/) (2022)
- [Extractive Adversarial Networks: High-Recall Explanations for Identifying Personal Attacks in Social Media Posts](https://mlsanity.com/paper/1809.01499/) (2018)
- [Sequential Explanations with Mental Model-Based Policies](https://mlsanity.com/paper/2007.09028/) (2020)
- [An Uncertainty-based Human-in-the-loop System for Industrial Tool Wear Analysis](https://mlsanity.com/paper/2007.07129/) (2020)
- [An Explainable AI System for Automated COVID-19 Assessment and Lesion Categorization from CT-scans](https://mlsanity.com/paper/2101.11943/) (2021)
- [On the Relationship Between Explanations, Fairness Perceptions, and Decisions](https://mlsanity.com/paper/2204.13156/) (2022)

## Built on by (most cited first, 29 of 29)

- [When combinations of humans and AI are useful: A systematic review and meta-analysis](https://mlsanity.com/paper/2405.06087/) (2024)
- [Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies](https://mlsanity.com/paper/2502.08554/) (2025)
- [Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction](https://mlsanity.com/paper/2303.09224/) (2023)
- [Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2209.11812/) (2022)
- [The Impact of Imperfect XAI on Human-AI Decision-Making](https://mlsanity.com/paper/2307.13566/) (2023)
- [Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making](https://mlsanity.com/paper/2401.07058/) (2024)
- [Using AI Uncertainty Quantification to Improve Human Decision-Making](https://mlsanity.com/paper/2309.10852/) (2023)
- [ALMANACS: A Simulatability Benchmark for Language Model Explainability](https://mlsanity.com/paper/2312.12747/) (2023)
- [AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventions](https://mlsanity.com/paper/2304.08804/) (2023)
- [Towards Effective Human-AI Decision-Making: The Role of Human Learning in Appropriate Reliance on AI Advice](https://mlsanity.com/paper/2310.02108/) (2023)
- [How explainable AI affects human performance: A systematic review of the behavioural consequences of saliency maps](https://mlsanity.com/paper/2404.16042/) (2024)
- [Explaining and Improving Information Complementarities in Multi-Agent Decision-making](https://mlsanity.com/paper/2502.06152/) (2025)
- [Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information](https://mlsanity.com/paper/2401.04729/) (2024)
- [A Survey of AI Reliance](https://mlsanity.com/paper/2408.03948/) (2024)
- [Generative AI and Creativity: A Systematic Literature Review and Meta-Analysis](https://mlsanity.com/paper/2505.17241/) (2025)
- [Transferring Domain Knowledge with (X)AI-Based Learning Systems](https://mlsanity.com/paper/2406.01329/) (2024)
- [Fine-Grained Appropriate Reliance: Human-AI Collaboration with a Multi-Step Transparent Decision Workflow for Complex Task Decomposition](https://mlsanity.com/paper/2501.10909/) (2025)
- [Explain To Decide: A Human-Centric Review on the Role of Explainable Artificial Intelligence in AI-assisted Decision Making](https://mlsanity.com/paper/2312.11507/) (2023)
- [Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of Human and Machine Explanations for Large Language Models](https://mlsanity.com/paper/2404.07725/) (2024)
- [Overcoming Anchoring Bias: The Potential of AI and XAI-based Decision Support](https://mlsanity.com/paper/2405.04972/) (2024)
- [ComplLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-making](https://mlsanity.com/paper/2602.19458/) (2026)
- [AIDE: Antithetical, Intent-based, and Diverse Example-Based Explanations](https://mlsanity.com/paper/2407.16010/) (2024)
- [The Effect of Explainable AI-based Decision Support on Human Task Performance: A Meta-Analysis](https://mlsanity.com/paper/2504.13858/) (2025)
- [An Empirical Examination of the Evaluative AI Framework](https://mlsanity.com/paper/2411.08583/) (2024)
- [Let people fail! Exploring the influence of explainable virtual and robotic agents in learning-by-doing tasks](https://mlsanity.com/paper/2411.10176/) (2024)
- [Navigating the Rashomon Effect: How Personalization Can Help Adjust Interpretable Machine Learning Models to Individual Users](https://mlsanity.com/paper/2505.07100/) (2025)
- [Development of Mental Models in Human-AI Collaboration: A Conceptual Framework](https://mlsanity.com/paper/2510.08104/) (2025)
- [Person-AI Bidirectional Fit - A Proof-Of-Concept Case Study Of Augmented Human-Ai Symbiosis In Management Decision-Making Process](https://mlsanity.com/paper/2511.13670/) (2025)
- [Human Decision-Making with Persuasive and Narrative LLM Explanations](https://mlsanity.com/paper/2605.23867/) (2026)

## Abstract (arXiv)

Research in artificial intelligence (AI)-assisted decision-making is experiencing tremendous growth with a constantly rising number of studies evaluating the effect of AI with and without techniques from the field of explainable AI (XAI) on human decision-making performance. However, as tasks and experimental setups vary due to different objectives, some studies report improved user decision-making performance through XAI, while others report only negligible effects. Therefore, in this article, we present an initial synthesis of existing research on XAI studies using a statistical meta-analysis to derive implications across existing research. We observe a statistically positive impact of XAI on users' performance. Additionally, the first results indicate that human-AI decision-making tends to yield better task performance on text data. However, we find no effect of explanations on users' performance compared to sole AI predictions. Our initial synthesis gives rise to future research investigating the underlying causes and contributes to further developing algorithms that effectively benefit human decision-makers by providing meaningful explanations.