Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
Not on the map. Why?
Abstract (arXiv)
In recent years, the field of explainable AI (XAI) has produced a vast collection of algorithms, providing a useful toolbox for researchers and practitioners to build XAI applications. With the rich application opportunities, explainability is believed to have moved beyond a demand by data scientists or researchers to comprehend the models they develop, to an essential requirement for people to trust and adopt AI deployed in numerous domains. However, explainability is an inherently human-centric property and the field is starting to embrace human-centered approaches. Human-computer interaction (HCI) research and user experience (UX) design in this area are becoming increasingly important. In this chapter, we begin with a high-level overview of the technical landscape of XAI algorithms, then selectively survey our own and other recent HCI works that take human-centered approaches to design, evaluate, and provide conceptual and methodological tools for XAI. We ask the question "what are human-centered approaches doing for XAI" and highlight three roles that they play in shaping XAI technologies by helping navigate, assess and expand the XAI toolbox: to drive technical choices by users' explainability needs, to uncover pitfalls of existing XAI methods and inform new methods, and to provide conceptual frameworks for human-compatible XAI.
Key facts. Cited by 123 papers in the map (data through 14 Oct 2026). Builds on 48 of its 109 references that are in the map. First posted on arXiv 20 Oct 2021.
Builds on
48 of 109 references.
References · 48
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
- 2018Datasheets for Datasetscited by 1,386
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
- 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
- 2018Model Cards for Model Reportingcited by 1,080
- 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
- 2016The Mythos of Model Interpretabilitycited by 721
36 more
- 2016Distribution-Free Predictive Inference For Regressioncited by 668
- 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 614
- 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
- 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 399
- 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
- 2018Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negativescited by 262
- 2018Manipulating and Measuring Model Interpretabilitycited by 256
- 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 253
- 2016Understanding Neural Networks through Representation Erasurecited by 224
- 2019Interpretable Counterfactual Explanations Guided by Prototypescited by 223
- 2016Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Modelscited by 213
- 2018Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learningcited by 207
- 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 190
- 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
- 2020Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systemscited by 139
- 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
- 2020Human-centered Explainable AI: Towards a Reflective Sociotechnical Approachcited by 90
- 2017Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciencescited by 85
- 2019Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptionscited by 81
- 2020Human Factors in Model Interpretability: Industry Practices, Challenges, and Needscited by 74
- 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
- 2020Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-makingcited by 59
- 2017Interpretable & Explorable Approximations of Black Box Modelscited by 54
- 2020The Pragmatic Turn in Explainable Artificial Intelligence (XAI)cited by 54
- 2018Stakeholders in Explainable AIcited by 48
- 2020CheXplain: Enabling Physicians to Explore and UnderstandData-Driven, AI-Enabled Medical Imaging Analysiscited by 40
- 2019Generalized Linear Rule Modelscited by 36
- 2018TED: Teaching AI to Explain its Decisionscited by 33
- 2019Why Does My Model Fail? Contrastive Local Explanations for Retail Forecastingcited by 31
- 2021Question-Driven Design Process for Explainable AI User Experiencescited by 31
- 2019A Grounded Interaction Protocol for Explainable Artificial Intelligencecited by 22
- 2021Uncertainty Quantification 360: A Holistic Toolkit for Quantifying and Communicating the Uncertainty of AIcited by 22
- 2020Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experiencecited by 20
- 2021From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidencecited by 17
- 2018Improving Simple Models with Confidence Profilescited by 12
- 2021Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoMLcited by 12
Built on by
Citing papers per year: 2021: 1; 2022: 22; 2023: 30; 2024: 28; 2025: 28; 2026: 14.
123 papers.
Sort:
By area:
- 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 123
- 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
- 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 85
- 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73
- 2023Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Makingcited by 67
- 2023Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision supportcited by 66
- 2022"Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interactioncited by 63
- 2023Rethinking Model Evaluation as Narrowing the Socio-Technical Gapcited by 56
- 2022Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contextscited by 53
- 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 53
- 2022Designing for Responsible Trust in AI Systems: A Communication Perspectivecited by 48
- 2022Investigating Explainability of Generative AI for Code through Scenario-based Designcited by 47
38 more
- 2023Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistencycited by 42
- 2023Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systemscited by 41
- 2022Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AIcited by 40
- 2025Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistenciescited by 39
- 2024Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Makingcited by 38
- 2023Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAIcited by 33
- 2023Selective Explanations: Leveraging Human Input to Align Explainable AIcited by 29
- 2023A Survey on Explainability of Graph Neural Networkscited by 29
- 2024Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task Decompositioncited by 29
- 2025Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checkingcited by 29
- 2025Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistantcited by 26
- 2023Dear XAI Community, We Need to Talk! Fundamental Misconceptions in Current XAI Researchcited by 22
- 2023The Impact of Imperfect XAI on Human-AI Decision-Makingcited by 21
- 2022Seamful XAI: Operationalizing Seamful Design in Explainable AIcited by 20
- 2023Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experiencecited by 18
- 2022On the Effect of Information Asymmetry in Human-AI Teamscited by 17
- 2023XAIR: A Framework of Explainable AI in Augmented Realitycited by 17
- 2023Learning by Self-Explainingcited by 17
- 2023Impact Of Explainable AI On Cognitive Load: Insights From An Empirical Studycited by 15
- 2022Deletion and Insertion Tests in Regression Modelscited by 14
- 2022From Black Boxes to Conversations: Incorporating XAI in a Conversational Agentcited by 14
- 2023FINER: Enhancing State-of-the-art Classifiers with Feature Attribution to Facilitate Security Analysiscited by 12
- 2024Challenges in Human-Agent Communicationcited by 12
- 2023Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Makingcited by 11
- 2024GPT for Games: An Updated Scoping Review (2020-2024)cited by 10
- 2022Beware the Rationalization Trap! When Language Model Explainability Diverges from our Mental Models of Languagecited by 9
- 2023Why is plausibility surprisingly problematic as an XAI criterion?cited by 9
- 2024On Evaluating Explanation Utility for Human-AI Decision Making in NLPcited by 9
- 2023Human-Centered Responsible Artificial Intelligence: Current & Future Trendscited by 8
- 2024Rethinking Explainable Machine Learning as Applied Statisticscited by 8
- 2024Incremental XAI: Memorable Understanding of AI with Incremental Explanationscited by 8
- 2024Understanding Large Language Model Behaviors through Interactive Counterfactual Generation and Analysiscited by 8
- 2022Mediators: Conversational Agents Explaining NLP Model Behaviorcited by 7
- 2024Unpacking Human-AI interactions: From interaction primitives to a design spacecited by 7
- 2024To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systemscited by 7
- 2025Evaluating Model Explanations without Ground Truthcited by 7
- 2025Improving Human Verification of LLM Reasoning through Interactive Explanation Interfacescited by 7
- 2022Social Construction of XAI: Do We Need One Definition to Rule Them All?cited by 6
50 most cited of 123 All 123 (JSON)
- 2026Understanding Operator Attitudes Toward AI-Supported Decision Making in Maritime Operations10 Sep 2026
- 2026XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?8 Sep 2026
- 2026Who Are You Explaining To? A Multi-Agent System for Audience-Aware XAI Narratives11 Aug 2026
- 2026XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection3 Jul 2026
- 2026Position: Explainability Research Must Prioritize Foundations over Ad-hoc Methods18 Jun 2026
- 2026SketchXplain: Intuitive Visual Explanations of Image Classifiers with Sketches16 Jun 2026
- 2026We Need Explanation Cards to Connect Explanation Algorithms to the Real World15 Jun 2026
- 2026"It became a self-fulfilling prophecy": How Lived Experiences are Entangled with AI Predictions in Menstrual Cycle Tracking Apps13 May 2026
- 2026Using Learning Theories to Evolve Human-Centered XAI: Future Perspectives and Challenges1 Apr 2026
- 2026Human-Centered Evaluation of an LLM-Based Process Modeling Copilot: A Mixed-Methods Study with Domain Experts13 Mar 2026
- 2026PONTE: Personalized Orchestration for Natural Language Trustworthy Explanations6 Mar 2026
- 2026Overseeing Agents Without Constant Oversight: Challenges and Opportunities18 Feb 2026
38 more
- 2026AI-exhibited Personality Traits Can Shape Human Self-concept through Conversations19 Jan 2026
- 2026Evaluating the Ability of Explanations to Disambiguate Models in a Rashomon Set13 Jan 2026
- 2025Explaining AI Without Code: A User Study on Explainable AI28 Dec 2025
- 2025Towards a Humanized Social-Media Ecosystem: AI-Augmented HCI Design Patterns for Safety, Agency & Well-Being8 Nov 2025
- 2025Improving Human Verification of LLM Reasoning through Interactive Explanation Interfaces27 Oct 2025
- 2025From Explainability to Action: A Generative Operational Framework for Integrating XAI in Clinical Mental Health Screening10 Oct 2025
- 2025Structuring Reasoning for Complex Rules Beyond Flat Representations1 Oct 2025
- 2025From Sea to System: Exploring User-Centered Explainable AI for Maritime Decision Support18 Sep 2025
- 2025Evalet: Evaluating Large Language Models through Functional Fragmentation14 Sep 2025
- 2025Exploring Human-AI Collaboration Using Mental Models of Early Adopters of Multi-Agent Generative AI Tools10 Sep 2025
- 2025GenTune: Toward Traceable Prompts to Improve Controllability of Image Refinement in Environment Design21 Aug 2025
- 2025Informative Post-Hoc Explanations Only Exist for Simple Functions15 Aug 2025
- 2025AIAP: A No-Code Workflow Builder for Non-Experts with Natural Language and Multi-Agent Collaboration4 Aug 2025
- 2025ADEPTS: A Capability Framework for Human-Centered Agent Design18 Jul 2025
- 2025Mind the XAI Gap: A Human-Centered LLM Framework for Democratizing Explainable AI13 Jun 2025
- 2025Evaluating Model Explanations without Ground Truth15 May 2025
- 2025See What I Mean? CUE: A Cognitive Model of Understanding Explanations9 May 2025
- 2025Exploring the Impact of Explainable AI and Cognitive Capabilities on Users' Decisions2 May 2025
- 2025From Questions to Insights: Exploring XAI Challenges Reported on Stack Overflow Questions3 Apr 2025
- 2025Investigating the Capabilities and Limitations of Machine Learning for Identifying Bias in English Language Data with Information and Heritage Professionals1 Apr 2025
- 2025Explainable AI-Based Interface System for Weather Forecasting Model1 Apr 2025
- 2025Conversational Explanations: Discussing Explainable AI with Non-AI Experts16 Feb 2025
- 2025A Survey on Human-Centered Evaluation of Explainable AI Methods in Clinical Decision Support Systems14 Feb 2025
- 2025Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checking13 Feb 2025
- 2025Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies12 Feb 2025
- 2025ExpProof : Operationalizing Explanations for Confidential Models with ZKPs6 Feb 2025
- 2025A Beautiful Mind: Principles and Strategies for AI-Augmented Human Reasoning5 Feb 2025
- 2025Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant29 Jan 2025
- 2025Policy alone is probably not the solution: A large-scale experiment on how developers struggle to design meaningful end-user explanations28 Jan 2025
- 2025Fine-Grained Appropriate Reliance: Human-AI Collaboration with a Multi-Step Transparent Decision Workflow for Complex Task Decomposition19 Jan 2025
- 2024Challenges in Human-Agent Communication28 Nov 2024
- 2024GPT for Games: An Updated Scoping Review (2020-2024)1 Nov 2024
- 2024Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting16 Oct 2024
- 2024Explaining Explaining26 Sep 2024
- 2024Interactive Example-based Explanations to Improve Health Professionals' Onboarding with AI for Human-AI Collaborative Decision Making24 Sep 2024
- 2024To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems22 Sep 2024
- 2024Predicting Trust In Autonomous Vehicles: Modeling Young Adult Psychosocial Traits, Risk-Benefit Attitudes, And Driving Factors With Machine Learning13 Sep 2024
- 2024What Did My Car Say? Impact of Autonomous Vehicle Explanation Errors and Driving Context On Comfort, Reliance, Satisfaction, and Driving Confidence9 Sep 2024
50 newest of 123 All 123 (JSON)
- 2024Challenges in Human-Agent Communicationcited by 12
- 2026Overseeing Agents Without Constant Oversight: Challenges and Opportunitiescited by 3
- 2025ADEPTS: A Capability Framework for Human-Centered Agent Designcited by 0
- 2026Who Are You Explaining To? A Multi-Agent System for Audience-Aware XAI Narrativescited by 0
- 2026Who Are You Explaining To? A Multi-Agent System for Audience-Aware XAI Narratives11 Aug 2026
- 2026Overseeing Agents Without Constant Oversight: Challenges and Opportunities18 Feb 2026
- 2025ADEPTS: A Capability Framework for Human-Centered Agent Design18 Jul 2025
- 2024Challenges in Human-Agent Communication28 Nov 2024
- 2023Rethinking Model Evaluation as Narrowing the Socio-Technical Gapcited by 56
- 2022From Black Boxes to Conversations: Incorporating XAI in a Conversational Agentcited by 14
- 2025Improving Human Verification of LLM Reasoning through Interactive Explanation Interfacescited by 7
- 2025Structuring Reasoning for Complex Rules Beyond Flat Representationscited by 0
- 2025Improving Human Verification of LLM Reasoning through Interactive Explanation Interfaces27 Oct 2025
- 2025Structuring Reasoning for Complex Rules Beyond Flat Representations1 Oct 2025
- 2023Rethinking Model Evaluation as Narrowing the Socio-Technical Gap1 Jun 2023
- 2022From Black Boxes to Conversations: Incorporating XAI in a Conversational Agent6 Sep 2022
- 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 123
- 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
- 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 85
- 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73
- 2023Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Makingcited by 67
- 2023Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision supportcited by 66
- 2022"Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interactioncited by 63
- 2022Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contextscited by 53
- 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 53
- 2022Designing for Responsible Trust in AI Systems: A Communication Perspectivecited by 48
- 2022Investigating Explainability of Generative AI for Code through Scenario-based Designcited by 47
- 2023Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistencycited by 42
12 most cited of 114 All 114 (JSON)
- 2026Understanding Operator Attitudes Toward AI-Supported Decision Making in Maritime Operations10 Sep 2026
- 2026XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?8 Sep 2026
- 2026XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection3 Jul 2026
- 2026SketchXplain: Intuitive Visual Explanations of Image Classifiers with Sketches16 Jun 2026
- 2026We Need Explanation Cards to Connect Explanation Algorithms to the Real World15 Jun 2026
- 2026"It became a self-fulfilling prophecy": How Lived Experiences are Entangled with AI Predictions in Menstrual Cycle Tracking Apps13 May 2026
- 2026Using Learning Theories to Evolve Human-Centered XAI: Future Perspectives and Challenges1 Apr 2026
- 2026Human-Centered Evaluation of an LLM-Based Process Modeling Copilot: A Mixed-Methods Study with Domain Experts13 Mar 2026
- 2026PONTE: Personalized Orchestration for Natural Language Trustworthy Explanations6 Mar 2026
- 2026AI-exhibited Personality Traits Can Shape Human Self-concept through Conversations19 Jan 2026
- 2026Evaluating the Ability of Explanations to Disambiguate Models in a Rashomon Set13 Jan 2026
- 2025Explaining AI Without Code: A User Study on Explainable AI28 Dec 2025
12 newest of 114 All 114 (JSON)