mlsanityData to 14 Oct 2026

What Do People Want to Know About Artificial Intelligence (AI)? The Importance of Answering End-User Questions to Explain Autonomous Vehicle (AV) Decisions

arXiv 2505.06428 ↗First posted 9 May 2025cs.HC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Improving end-users' understanding of decisions made by autonomous vehicles (AVs) driven by artificial intelligence (AI) can improve utilization and acceptance of AVs. However, current explanation mechanisms primarily help AI researchers and engineers in debugging and monitoring their AI systems, and may not address the specific questions of end-users, such as passengers, about AVs in various scenarios. In this paper, we conducted two user studies to investigate questions that potential AV passengers might pose while riding in an AV and evaluate how well answers to those questions improve their understanding of AI-driven AV decisions. Our initial formative study identified a range of questions about AI in autonomous driving that existing explanation mechanisms do not readily address. Our second study demonstrated that interactive text-based explanations effectively improved participants' comprehension of AV decisions compared to simply observing AV decisions. These findings inform the design of interactions that motivate end-users to engage with and inquire about the reasoning behind AI-driven AV decisions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 133 references that are in the map. First posted on arXiv 9 May 2025.

Builds on

42 of 133 references.

References · 42

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  3. 2015VQA: Visual Question Answeringcited by 2,506
  4. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  5. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  6. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 614
  7. 2019The What-If Tool: Interactive Probing of Machine Learning Modelscited by 207
  8. 2019Model-Agnostic Counterfactual Explanations for Consequential Decisionscited by 204
  9. 2016Generating Visual Explanationscited by 158
  10. 2020Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systemscited by 139
  11. 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
  12. 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
30 more
  1. 2019"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanationscited by 111
  2. 2022Explanations Can Reduce Overreliance on AI Systems During Decision-Makingcited by 111
  3. 2019Towards Explainable Artificial Intelligencecited by 106
  4. 2020Human-centered Explainable AI: Towards a Reflective Sociotechnical Approachcited by 90
  5. 2020Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Studycited by 86
  6. 2021Explanations in Autonomous Driving: A Surveycited by 81
  7. 2022TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversationscited by 66
  8. 2017ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Modelscited by 56
  9. 2022Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learningcited by 53
  10. 2024Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Reviewcited by 53
  11. 2023In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Makingcited by 50
  12. 2018Explanation Methods in Deep Learning: Users, Values, Concerns and Challengescited by 49
  13. 2021The Who in XAI: How AI Background Shapes Perceptions of AI Explanationscited by 48
  14. 2018The Challenge of Crafting Intelligible Intelligencecited by 47
  15. 2020How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labelscited by 39
  16. 2023Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Carecited by 37
  17. 2022Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Makingcited by 33
  18. 2022Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theorycited by 32
  19. 2025Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistantcited by 26
  20. 2022Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Makingcited by 25
  21. 2023The Impact of Imperfect XAI on Human-AI Decision-Makingcited by 21
  22. 2021Conceptualising Contestability: Perspectives on Contesting Algorithmic Decisionscited by 18
  23. 2022Symphony: Composing Interactive Interfaces for Machine Learningcited by 18
  24. 2022From Black Boxes to Conversations: Incorporating XAI in a Conversational Agentcited by 14
  25. 2019Look Who's Talking Now: Implications of AV's Explanations on Driver's Trust, AV Preference, Anxiety and Mental Workloadcited by 12
  26. 2020What Would You Ask the Machine Learning Model? Identification of User Needs for Model Explanations Based on Human-Model Conversationscited by 11
  27. 2021An Aligned Rank Transform Procedure for Multifactor Contrast Testscited by 11
  28. 2022"If it didn't happen, why would I change my decision?": How Judges Respond to Counterfactual Explanations for the Public Safety Assessmentcited by 6
  29. 2024Advancing Explainable Autonomous Vehicle Systems: A Comprehensive Review and Research Roadmapcited by 6
  30. 2019Lessons from Oz: Design Guidelines for Automotive Conversational User Interfacescited by 3

Built on by

Not cited yet.