Explainability in Human-Agent Systems
Not on the map. Why?
Abstract (arXiv)
This paper presents a taxonomy of explainability in Human-Agent Systems. We consider fundamental questions about the Why, Who, What, When and How of explainability. First, we define explainability, and its relationship to the related terms of interpretability, transparency, explicitness, and faithfulness. These definitions allow us to answer why explainability is needed in the system, whom it is geared to and what explanations can be generated to meet this need. We then consider when the user should be presented with this information. Last, we consider how objective and subjective measures can be used to evaluate the entire system. This last question is the most encompassing as it will need to evaluate all other issues regarding explainability.
Key facts. Cited by 34 papers in the map (data through 13 Oct 2026). First posted on arXiv 17 Apr 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 10; 2022: 5; 2023: 5; 2024: 8; 2025: 5; 2026: 1.
34 papers.
Sort:
By area:
- 2021Trustworthy AI: From Principles to Practicescited by 206
- 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
- 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
- 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
- 2021Argumentative XAI: A Surveycited by 66
- 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systemscited by 59
- 2021The Who in XAI: How AI Background Shapes Perceptions of AI Explanationscited by 48
- 2021Question-Driven Design Process for Explainable AI User Experiencescited by 31
- 2024Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectivescited by 30
- 2022Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunitiescited by 25
- 2023A Critical Survey on Fairness Benefits of Explainable AIcited by 16
- 2021Explainable Autonomous Robots: A Survey and Perspectivecited by 15
22 more
- 2021Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligencecited by 13
- 2023Ethical Framework for Harnessing the Power of AI in Healthcare and Beyondcited by 6
- 2024What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blockscited by 6
- 2023A New Perspective on Evaluation Methods for Explainable Artificial Intelligence (XAI)cited by 5
- 2022Justifying Social-Choice Mechanism Outcome for Improving Participant Satisfactioncited by 4
- 2021A Hybrid Approach for an Interpretable and Explainable Intrusion Detection Systemcited by 3
- 2024Multi-stakeholder Perspective on Responsible Artificial Intelligence and Acceptability in Educationcited by 2
- 2024Enhancing Trust in Autonomous Agents: An Architecture for Accountability and Explainability through Blockchain and Large Language Modelscited by 2
- 2024Explainable Interface for Human-Autonomy Teaming: A Surveycited by 2
- 2025Inductive Learning of Robot Task Knowledge from Raw Data and Online Expert Feedbackcited by 2
- 2025Not All Explanations are Created Equal: Investigating the Pitfalls of Current XAI Evaluationcited by 2
- 2022An Urban Population Health Observatory for Disease Causal Pathway Analysis and Decision Support: Underlying Explainable Artificial Intelligence Modelcited by 1
- 2022Interpretable Explainability in Facial Emotion Recognition and Gamification for Data Collectioncited by 1
- 2024Structure and Reduction of MCTS for Explainable-AIcited by 1
- 2024Semifactual Explanations for Reinforcement Learningcited by 1
- 2025Towards LLM-generated explanations for Component-based Knowledge Graph Question Answering Systemscited by 1
- 2021Local Explanations for Clinical Search Engine resultscited by 0
- 2022Introspection-based Explainable Reinforcement Learning in Episodic and Non-episodic Scenarioscited by 0
- 2023Fidelity-Induced Interpretable Policy Extraction for Reinforcement Learningcited by 0
- 2024Explaining Explainingcited by 0
- 2025Uncertainty Awareness and Trust in Explainable AI- On Trust Calibration using Local and Global Explanationscited by 0
- 2026Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?cited by 0
- 2026Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?25 May 2026
- 2025Not All Explanations are Created Equal: Investigating the Pitfalls of Current XAI Evaluation27 Sep 2025
- 2025Uncertainty Awareness and Trust in Explainable AI- On Trust Calibration using Local and Global Explanations10 Sep 2025
- 2025Towards LLM-generated explanations for Component-based Knowledge Graph Question Answering Systems20 Aug 2025
- 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems4 Jun 2025
- 2025Inductive Learning of Robot Task Knowledge from Raw Data and Online Expert Feedback13 Jan 2025
- 2024Explaining Explaining26 Sep 2024
- 2024Semifactual Explanations for Reinforcement Learning9 Sep 2024
- 2024Structure and Reduction of MCTS for Explainable-AI10 Aug 2024
- 2024Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectives20 Jul 2024
- 2024Explainable Interface for Human-Autonomy Teaming: A Survey4 May 2024
- 2024What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks19 Mar 2024
22 more
- 2024Enhancing Trust in Autonomous Agents: An Architecture for Accountability and Explainability through Blockchain and Large Language Models14 Mar 2024
- 2024Multi-stakeholder Perspective on Responsible Artificial Intelligence and Acceptability in Education22 Feb 2024
- 2023A Critical Survey on Fairness Benefits of Explainable AI15 Oct 2023
- 2023Fidelity-Induced Interpretable Policy Extraction for Reinforcement Learning12 Sep 2023
- 2023Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond31 Aug 2023
- 2023A New Perspective on Evaluation Methods for Explainable Artificial Intelligence (XAI)26 Jul 2023
- 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey10 Jul 2023
- 2022Introspection-based Explainable Reinforcement Learning in Episodic and Non-episodic Scenarios23 Nov 2022
- 2022Interpretable Explainability in Facial Emotion Recognition and Gamification for Data Collection9 Nov 2022
- 2022An Urban Population Health Observatory for Disease Causal Pathway Analysis and Decision Support: Underlying Explainable Artificial Intelligence Model26 Jul 2022
- 2022Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunities13 Jul 2022
- 2022Justifying Social-Choice Mechanism Outcome for Improving Participant Satisfaction24 May 2022
- 2021Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligence29 Dec 2021
- 2021A Hybrid Approach for an Interpretable and Explainable Intrusion Detection System19 Nov 2021
- 2021Local Explanations for Clinical Search Engine results19 Oct 2021
- 2021Trustworthy AI: From Principles to Practices4 Oct 2021
- 2021The Who in XAI: How AI Background Shapes Perceptions of AI Explanations28 Jul 2021
- 2021Argumentative XAI: A Survey24 May 2021
- 2021Explainable Autonomous Robots: A Survey and Perspective6 May 2021
- 2021Question-Driven Design Process for Explainable AI User Experiences8 Apr 2021
- 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research15 Feb 2021
- 2021Explainability of deep vision-based autonomous driving systems: Review and challenges13 Jan 2021
- 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systemscited by 59
- 2024Enhancing Trust in Autonomous Agents: An Architecture for Accountability and Explainability through Blockchain and Large Language Modelscited by 2
- 2026Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?cited by 0
- 2026Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?25 May 2026
- 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems4 Jun 2025
- 2024Enhancing Trust in Autonomous Agents: An Architecture for Accountability and Explainability through Blockchain and Large Language Models14 Mar 2024
- 2024Semifactual Explanations for Reinforcement Learningcited by 1
- 2024Semifactual Explanations for Reinforcement Learning9 Sep 2024
- 2021Trustworthy AI: From Principles to Practicescited by 206
- 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
- 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
- 2021Argumentative XAI: A Surveycited by 66
- 2021The Who in XAI: How AI Background Shapes Perceptions of AI Explanationscited by 48
- 2021Question-Driven Design Process for Explainable AI User Experiencescited by 31
- 2024Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectivescited by 30
- 2022Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunitiescited by 25
- 2023A Critical Survey on Fairness Benefits of Explainable AIcited by 16
- 2021Explainable Autonomous Robots: A Survey and Perspectivecited by 15
- 2021Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligencecited by 13
- 2023Ethical Framework for Harnessing the Power of AI in Healthcare and Beyondcited by 6
12 most cited of 29 All 29 (JSON)
- 2025Not All Explanations are Created Equal: Investigating the Pitfalls of Current XAI Evaluation27 Sep 2025
- 2025Uncertainty Awareness and Trust in Explainable AI- On Trust Calibration using Local and Global Explanations10 Sep 2025
- 2025Towards LLM-generated explanations for Component-based Knowledge Graph Question Answering Systems20 Aug 2025
- 2025Inductive Learning of Robot Task Knowledge from Raw Data and Online Expert Feedback13 Jan 2025
- 2024Explaining Explaining26 Sep 2024
- 2024Structure and Reduction of MCTS for Explainable-AI10 Aug 2024
- 2024Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectives20 Jul 2024
- 2024Explainable Interface for Human-Autonomy Teaming: A Survey4 May 2024
- 2024What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks19 Mar 2024
- 2024Multi-stakeholder Perspective on Responsible Artificial Intelligence and Acceptability in Education22 Feb 2024
- 2023A Critical Survey on Fairness Benefits of Explainable AI15 Oct 2023
- 2023Fidelity-Induced Interpretable Policy Extraction for Reinforcement Learning12 Sep 2023
12 newest of 29 All 29 (JSON)