# Towards Reconciling Usability and Usefulness of Explainable AI Methodologies arXiv: https://arxiv.org/abs/2301.05347 ยท HTML: https://mlsanity.com/paper/2301.05347/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Builds on 35 of its 69 references that are in the map. - First posted on arXiv 13 Jan 2023. ## Builds on (most cited first, 30 of 35) - [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016) - [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013) - [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017) - [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017) - [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018) - [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019) - [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019) - [Understanding Neural Networks Through Deep Visualization](https://mlsanity.com/paper/1506.06579/) (2015) - [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017) - [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020) - [Metrics for Explainable AI: Challenges and Prospects](https://mlsanity.com/paper/1812.04608/) (2018) - [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020) - [Explainable Prediction of Medical Codes from Clinical Text](https://mlsanity.com/paper/1802.05695/) (2018) - [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017) - [Explainable Reinforcement Learning Through a Causal Lens](https://mlsanity.com/paper/1905.10958/) (2019) - [Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI](https://mlsanity.com/paper/2010.07487/) (2020) - [Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model](https://mlsanity.com/paper/1511.01644/) (2015) - [What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use](https://mlsanity.com/paper/1905.05134/) (2019) - [Expanding Explainability: Towards Social Transparency in AI systems](https://mlsanity.com/paper/2101.04719/) (2021) - [Human-centered Explainable AI: Towards a Reflective Sociotechnical Approach](https://mlsanity.com/paper/2002.01092/) (2020) - [Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptions](https://mlsanity.com/paper/1901.03729/) (2019) - [Contrastive Explanation: A Structural-Model Approach](https://mlsanity.com/paper/1811.03163/) (2018) - [The Emerging Landscape of Explainable AI Planning and Decision Making](https://mlsanity.com/paper/2002.11697/) (2020) - [Interpretable Credit Application Predictions With Counterfactual Explanations](https://mlsanity.com/paper/1811.05245/) (2018) - [Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery](https://mlsanity.com/paper/2101.01625/) (2021) - [The Utility of Explainable AI in Ad Hoc Human-Machine Teaming](https://mlsanity.com/paper/2209.03943/) (2022) - [Exploring Computational User Models for Agent Policy Summarization](https://mlsanity.com/paper/1905.13271/) (2019) - [Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference](https://mlsanity.com/paper/1808.03894/) (2018) - [FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation](https://mlsanity.com/paper/2012.15482/) (2020) - [ExSum: From Local Explanations to Model Understanding](https://mlsanity.com/paper/2205.00130/) (2022) ## Built on by (most cited first, 2 of 2) - [Towards Balancing Preference and Performance through Adaptive Personalized Explainability](https://mlsanity.com/paper/2504.13856/) (2025) - [Could Large Language Models work as Post-hoc Explainability Tools in Credit Risk Models?](https://mlsanity.com/paper/2602.18895/) (2026) ## Abstract (arXiv) Interactive Artificial Intelligence (AI) agents are becoming increasingly prevalent in society. However, application of such systems without understanding them can be problematic. Black-box AI systems can lead to liability and accountability issues when they produce an incorrect decision. Explainable AI (XAI) seeks to bridge the knowledge gap, between developers and end-users, by offering insights into how an AI algorithm functions. Many modern algorithms focus on making the AI model "transparent", i.e. unveil the inherent functionality of the agent in a simpler format. However, these approaches do not cater to end-users of these systems, as users may not possess the requisite knowledge to understand these explanations in a reasonable amount of time. Therefore, to be able to develop suitable XAI methods, we need to understand the factors which influence subjective perception and objective usability. In this paper, we present a novel user-study which studies four differing XAI modalities commonly employed in prior work for explaining AI behavior, i.e. Decision Trees, Text, Programs. We study these XAI modalities in the context of explaining the actions of a self-driving car on a highway, as driving is an easily understandable real-world task and self-driving cars is a keen area of interest within the AI community. Our findings highlight internal consistency issues wherein participants perceived language explanations to be significantly more usable, however participants were better able to objectively understand the decision making process of the car through a decision tree explanation. Our work also provides further evidence of importance of integrating user-specific and situational criteria into the design of XAI systems. Our findings show that factors such as computer science experience, and watching the car succeed or fail can impact the perception and usefulness of the explanation.