mlsanityData to 7 Oct 2026

Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations

arXiv 2210.11584 ↗First posted 20 Oct 2022cs.AI cs.HCcited by 84

Not on the map. Why?

Abstract (arXiv)

Explainable AI (XAI) is widely viewed as a sine qua non for ever-expanding AI research. A better understanding of the needs of XAI users, as well as human-centered evaluations of explainable models are both a necessity and a challenge. In this paper, we explore how HCI and AI researchers conduct user studies in XAI applications based on a systematic literature review. After identifying and thoroughly analyzing 97core papers with human-based XAI evaluations over the past five years, we categorize them along the measured characteristics of explanatory methods, namely trust, understanding, usability, and human-AI collaboration performance. Our research shows that XAI is spreading more rapidly in certain application domains, such as recommender systems than in others, but that user evaluations are still rather sparse and incorporate hardly any insights from cognitive or social sciences. Based on a comprehensive discussion of best practices, i.e., common models, design choices, and measures in user studies, we propose practical guidelines on designing and conducting user studies for XAI researchers and practitioners. Lastly, this survey also highlights several open research directions, particularly linking psychological science and human-centered XAI.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Builds on 102 of its 290 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

102 of 290 references.

References · 102

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  6. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  9. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  10. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  11. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  12. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
38 more
  1. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  2. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  3. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
  4. 2020Concept Bottleneck Modelscited by 1,022
  5. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  6. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  7. 2019Self-labelling via simultaneous clustering and representation learningcited by 525
  8. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  9. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  10. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
  11. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
  12. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  13. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 343
  14. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
  15. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  16. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  17. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  18. 2018Metrics for Explainable AI: Challenges and Prospectscited by 257
  19. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  20. 2020Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Surveycited by 252
  21. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  22. 2020A Survey on the Explainability of Supervised Machine Learningcited by 225
  23. 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 189
  24. 2019Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practicescited by 170
  25. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
  26. 2020Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systemscited by 138
  27. 2018i-RevNet: Deep Invertible Networkscited by 137
  28. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  29. 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
  30. 2021Human-Centered Explainable AI (XAI): From Algorithms to User Experiencescited by 123
  31. 2019Sanity Checks for Saliency Metricscited by 119
  32. 2019Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Makingcited by 109
  33. 2019Towards Explainable Artificial Intelligencecited by 106
  34. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  35. 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 100
  36. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  37. 2020Getting a CLUE: A Method for Explaining Uncertainty Estimatescited by 92
  38. 2018'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisionscited by 91

50 most cited of 102 All 102 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 27; 2025: 31; 2026: 20.

84 papers.

Sort:

By area:

  1. 2024What Large Language Models Know and What People Think They Knowcited by 109
  2. 2024shapiq: Shapley Interactions for Machine Learningcited by 42
  3. 2025Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistantcited by 26
  4. 2025From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centeredcited by 21
  5. 2023Tell Me a Story! Narrative-Driven XAI with Large Language Modelscited by 20
  6. 2025HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluationcited by 19
  7. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  8. 2024Is My Data in Your AI? Membership Inference Test (MINT) applied to Face Biometricscited by 12
  9. 2024Explainable Predictive Maintenance: A Survey of Current Methods, Challenges and Opportunitiescited by 10
  10. 2024GPT for Games: An Updated Scoping Review (2020-2024)cited by 10
  11. 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Futurecited by 9
  12. 2025CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantificationcited by 9
38 more
  1. 2025VideoMiner: Iteratively Grounding Key Frames of Hour-Long Videos via Tree-based Group Relative Policy Optimizationcited by 9
  2. 2024Attention Mechanisms Don't Learn Additive Models: Rethinking Feature Importance for Transformerscited by 8
  3. 2024Evaluating Explanations Through LLMs: Beyond Traditional User Studiescited by 8
  4. 2023A Survey on Trustworthy Edge Intelligence: From Security and Reliability To Transparency and Sustainabilitycited by 7
  5. 2024Classification Metrics for Image Explanations: Towards Building Reliable XAI-Evaluationscited by 7
  6. 2024XEdgeAI: A Human-centered Industrial Inspection Framework with Data-centric Explainable Edge AI Approachcited by 7
  7. 2025Which LIME should I trust? Concepts, Challenges, and Solutionscited by 7
  8. 2025Improving Human Verification of LLM Reasoning through Interactive Explanation Interfacescited by 7
  9. 2023Human Attention-Guided Explainable Artificial Intelligence for Computer Vision Modelscited by 6
  10. 2024Survey of Computerized Adaptive Testing: A Machine Learning Perspectivecited by 6
  11. 2024How explainable AI affects human performance: A systematic review of the behavioural consequences of saliency mapscited by 6
  12. 2025Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Taskscited by 6
  13. 2025Explaining Similarity in Vision-Language Encoders with Weighted Banzhaf Interactionscited by 6
  14. 2024Aggregated Attributions for Explanatory Analysis of 3D Segmentation Modelscited by 5
  15. 2025Class-Dependent Perturbation Effects in Evaluating Time Series Attributionscited by 5
  16. 2025MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answeringcited by 5
  17. 2024Evaluating the Explainability of Attributes and Prototypes for a Medical Classification Modelcited by 4
  18. 2024Efficient and Accurate Explanation Estimation with Distribution Compressioncited by 4
  19. 2024AI Readiness in Healthcare through Storytelling XAIcited by 4
  20. 2025Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency mapscited by 4
  21. 2023Towards Interpretability in Audio and Visual Affective Machine Learning: A Reviewcited by 3
  22. 2023An HCAI Methodological Framework (HCAI-MF): Putting It Into Action to Enable Human-Centered AIcited by 3
  23. 2024Towards User-Focused Research in Training Data Attribution for Human-Centered Explainable AIcited by 3
  24. 2024Toward A Causal Framework for Modeling Perceptioncited by 2
  25. 2024Selective Explanationscited by 2
  26. 2024A Sim2Real Approach for Identifying Task-Relevant Properties in Interpretable Machine Learningcited by 2
  27. 2025Why Do Class-Dependent Evaluation Effects Occur with Time Series Feature Attributions? A Synthetic Data Investigationcited by 2
  28. 2025FunnyNodules: A Customizable Medical Dataset Tailored for Evaluating Explainable AIcited by 2
  29. 2026When AI Persuades: Adversarial Explanation Attacks on Human Trust in AI-Assisted Decision Makingcited by 2
  30. 2023I-CEE: Tailoring Explanations of Image Classification Models to User Expertisecited by 1
  31. 2024Data Science Principles for Interpretable and Explainable AIcited by 1
  32. 2024Inpainting the Gaps: A Novel Framework for Evaluating Explanation Methods in Vision Transformerscited by 1
  33. 2024AIDE: Antithetical, Intent-based, and Diverse Example-Based Explanationscited by 1
  34. 2025Explainable Multimodal Depression Recognition in Clinical Interviews via PHQ-Aligned Symptom Summarizationcited by 1
  35. 2025Exploring the Impact of Explainable AI and Cognitive Capabilities on Users' Decisionscited by 1
  36. 2025Can Agents Judge Systematic Reviews Like Humans? Evaluating SLRs with LLM-based Multi-Agent Systemcited by 1
  37. 2026Functional Decomposition and Shapley Interactions for Interpreting Survival Modelscited by 1
  38. 2026PONTE: Personalized Orchestration for Natural Language Trustworthy Explanationscited by 1

50 most cited of 84 All 84 (JSON)