mlsanityData to 14 Oct 2026

Human-Centered Explainable AI (XAI): From Algorithms to User Experiences

arXiv 2110.10790 ↗First posted 20 Oct 2021cs.AI cs.HCcited by 123

Not on the map. Why?

Abstract (arXiv)

In recent years, the field of explainable AI (XAI) has produced a vast collection of algorithms, providing a useful toolbox for researchers and practitioners to build XAI applications. With the rich application opportunities, explainability is believed to have moved beyond a demand by data scientists or researchers to comprehend the models they develop, to an essential requirement for people to trust and adopt AI deployed in numerous domains. However, explainability is an inherently human-centric property and the field is starting to embrace human-centered approaches. Human-computer interaction (HCI) research and user experience (UX) design in this area are becoming increasingly important. In this chapter, we begin with a high-level overview of the technical landscape of XAI algorithms, then selectively survey our own and other recent HCI works that take human-centered approaches to design, evaluate, and provide conceptual and methodological tools for XAI. We ask the question "what are human-centered approaches doing for XAI" and highlight three roles that they play in shaping XAI technologies by helping navigate, assess and expand the XAI toolbox: to drive technical choices by users' explainability needs, to uncover pitfalls of existing XAI methods and inform new methods, and to provide conceptual frameworks for human-compatible XAI.

Key facts. Cited by 123 papers in the map (data through 14 Oct 2026). Builds on 48 of its 109 references that are in the map. First posted on arXiv 20 Oct 2021.

Builds on

48 of 109 references.

References · 48

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  4. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  5. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  6. 2018Datasheets for Datasetscited by 1,386
  7. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  8. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  9. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  10. 2018Model Cards for Model Reportingcited by 1,080
  11. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  12. 2016The Mythos of Model Interpretabilitycited by 721
36 more
  1. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  2. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 614
  3. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  4. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 399
  5. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  6. 2018Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negativescited by 262
  7. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  8. 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 253
  9. 2016Understanding Neural Networks through Representation Erasurecited by 224
  10. 2019Interpretable Counterfactual Explanations Guided by Prototypescited by 223
  11. 2016Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Modelscited by 213
  12. 2018Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learningcited by 207
  13. 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 190
  14. 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
  15. 2020Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systemscited by 139
  16. 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
  17. 2020Human-centered Explainable AI: Towards a Reflective Sociotechnical Approachcited by 90
  18. 2017Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciencescited by 85
  19. 2019Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptionscited by 81
  20. 2020Human Factors in Model Interpretability: Industry Practices, Challenges, and Needscited by 74
  21. 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
  22. 2020Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-makingcited by 59
  23. 2017Interpretable & Explorable Approximations of Black Box Modelscited by 54
  24. 2020The Pragmatic Turn in Explainable Artificial Intelligence (XAI)cited by 54
  25. 2018Stakeholders in Explainable AIcited by 48
  26. 2020CheXplain: Enabling Physicians to Explore and UnderstandData-Driven, AI-Enabled Medical Imaging Analysiscited by 40
  27. 2019Generalized Linear Rule Modelscited by 36
  28. 2018TED: Teaching AI to Explain its Decisionscited by 33
  29. 2019Why Does My Model Fail? Contrastive Local Explanations for Retail Forecastingcited by 31
  30. 2021Question-Driven Design Process for Explainable AI User Experiencescited by 31
  31. 2019A Grounded Interaction Protocol for Explainable Artificial Intelligencecited by 22
  32. 2021Uncertainty Quantification 360: A Holistic Toolkit for Quantifying and Communicating the Uncertainty of AIcited by 22
  33. 2020Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experiencecited by 20
  34. 2021From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidencecited by 17
  35. 2018Improving Simple Models with Confidence Profilescited by 12
  36. 2021Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoMLcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 22; 2023: 30; 2024: 28; 2025: 28; 2026: 14.

123 papers.

Sort:

By area:

  1. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 123
  2. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  3. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 85
  4. 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73
  5. 2023Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Makingcited by 67
  6. 2023Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision supportcited by 66
  7. 2022"Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interactioncited by 63
  8. 2023Rethinking Model Evaluation as Narrowing the Socio-Technical Gapcited by 56
  9. 2022Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contextscited by 53
  10. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 53
  11. 2022Designing for Responsible Trust in AI Systems: A Communication Perspectivecited by 48
  12. 2022Investigating Explainability of Generative AI for Code through Scenario-based Designcited by 47
38 more
  1. 2023Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistencycited by 42
  2. 2023Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systemscited by 41
  3. 2022Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AIcited by 40
  4. 2025Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistenciescited by 39
  5. 2024Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Makingcited by 38
  6. 2023Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAIcited by 33
  7. 2023Selective Explanations: Leveraging Human Input to Align Explainable AIcited by 29
  8. 2023A Survey on Explainability of Graph Neural Networkscited by 29
  9. 2024Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task Decompositioncited by 29
  10. 2025Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checkingcited by 29
  11. 2025Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistantcited by 26
  12. 2023Dear XAI Community, We Need to Talk! Fundamental Misconceptions in Current XAI Researchcited by 22
  13. 2023The Impact of Imperfect XAI on Human-AI Decision-Makingcited by 21
  14. 2022Seamful XAI: Operationalizing Seamful Design in Explainable AIcited by 20
  15. 2023Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experiencecited by 18
  16. 2022On the Effect of Information Asymmetry in Human-AI Teamscited by 17
  17. 2023XAIR: A Framework of Explainable AI in Augmented Realitycited by 17
  18. 2023Learning by Self-Explainingcited by 17
  19. 2023Impact Of Explainable AI On Cognitive Load: Insights From An Empirical Studycited by 15
  20. 2022Deletion and Insertion Tests in Regression Modelscited by 14
  21. 2022From Black Boxes to Conversations: Incorporating XAI in a Conversational Agentcited by 14
  22. 2023FINER: Enhancing State-of-the-art Classifiers with Feature Attribution to Facilitate Security Analysiscited by 12
  23. 2024Challenges in Human-Agent Communicationcited by 12
  24. 2023Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Makingcited by 11
  25. 2024GPT for Games: An Updated Scoping Review (2020-2024)cited by 10
  26. 2022Beware the Rationalization Trap! When Language Model Explainability Diverges from our Mental Models of Languagecited by 9
  27. 2023Why is plausibility surprisingly problematic as an XAI criterion?cited by 9
  28. 2024On Evaluating Explanation Utility for Human-AI Decision Making in NLPcited by 9
  29. 2023Human-Centered Responsible Artificial Intelligence: Current & Future Trendscited by 8
  30. 2024Rethinking Explainable Machine Learning as Applied Statisticscited by 8
  31. 2024Incremental XAI: Memorable Understanding of AI with Incremental Explanationscited by 8
  32. 2024Understanding Large Language Model Behaviors through Interactive Counterfactual Generation and Analysiscited by 8
  33. 2022Mediators: Conversational Agents Explaining NLP Model Behaviorcited by 7
  34. 2024Unpacking Human-AI interactions: From interaction primitives to a design spacecited by 7
  35. 2024To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systemscited by 7
  36. 2025Evaluating Model Explanations without Ground Truthcited by 7
  37. 2025Improving Human Verification of LLM Reasoning through Interactive Explanation Interfacescited by 7
  38. 2022Social Construction of XAI: Do We Need One Definition to Rule Them All?cited by 6

50 most cited of 123 All 123 (JSON)