# What Do People Want to Know About Artificial Intelligence (AI)? The Importance of Answering End-User Questions to Explain Autonomous Vehicle (AV) Decisions

arXiv: https://arxiv.org/abs/2505.06428 · HTML: https://mlsanity.com/paper/2505.06428/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 42 of its 133 references that are in the map.
- First posted on arXiv 9 May 2025.

## Builds on (most cited first, 30 of 42)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Model-Agnostic Counterfactual Explanations for Consequential Decisions](https://mlsanity.com/paper/1905.11190/) (2019)
- [Generating Visual Explanations](https://mlsanity.com/paper/1603.08507/) (2016)
- [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020)
- [Expanding Explainability: Towards Social Transparency in AI systems](https://mlsanity.com/paper/2101.04719/) (2021)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- [Explanations Can Reduce Overreliance on AI Systems During Decision-Making](https://mlsanity.com/paper/2212.06823/) (2022)
- ["How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations](https://mlsanity.com/paper/1911.06473/) (2019)
- [Towards Explainable Artificial Intelligence](https://mlsanity.com/paper/1909.12072/) (2019)
- [Human-centered Explainable AI: Towards a Reflective Sociotechnical Approach](https://mlsanity.com/paper/2002.01092/) (2020)
- [Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Study](https://mlsanity.com/paper/2002.00772/) (2020)
- [Explanations in Autonomous Driving: A Survey](https://mlsanity.com/paper/2103.05154/) (2021)
- [TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations](https://mlsanity.com/paper/2207.04154/) (2022)
- [ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models](https://mlsanity.com/paper/1704.01942/) (2017)
- [Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning](https://mlsanity.com/paper/2202.05402/) (2022)
- [Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review](https://mlsanity.com/paper/2402.10086/) (2024)
- [In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Making](https://mlsanity.com/paper/2305.07722/) (2023)
- [Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges](https://mlsanity.com/paper/1803.07517/) (2018)
- [The Who in XAI: How AI Background Shapes Perceptions of AI Explanations](https://mlsanity.com/paper/2107.13509/) (2021)
- [The Challenge of Crafting Intelligible Intelligence](https://mlsanity.com/paper/1803.04263/) (2018)
- [How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labels](https://mlsanity.com/paper/2008.11721/) (2020)
- [Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Care](https://mlsanity.com/paper/2302.00096/) (2023)
- [Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Making](https://mlsanity.com/paper/2208.07960/) (2022)
- [Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory](https://mlsanity.com/paper/2205.05057/) (2022)

## Abstract (arXiv)

Improving end-users' understanding of decisions made by autonomous vehicles (AVs) driven by artificial intelligence (AI) can improve utilization and acceptance of AVs. However, current explanation mechanisms primarily help AI researchers and engineers in debugging and monitoring their AI systems, and may not address the specific questions of end-users, such as passengers, about AVs in various scenarios. In this paper, we conducted two user studies to investigate questions that potential AV passengers might pose while riding in an AV and evaluate how well answers to those questions improve their understanding of AI-driven AV decisions. Our initial formative study identified a range of questions about AI in autonomous driving that existing explanation mechanisms do not readily address. Our second study demonstrated that interactive text-based explanations effectively improved participants' comprehension of AV decisions compared to simply observing AV decisions. These findings inform the design of interactions that motivate end-users to engage with and inquire about the reasoning behind AI-driven AV decisions.