mlsanityData to 14 Oct 2026

May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainability

arXiv 2309.13965 ↗First posted 25 Sep 2023cs.HC cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Research in explainable AI (XAI) aims to provide insights into the decision-making process of opaque AI models. To date, most XAI methods offer one-off and static explanations, which cannot cater to the diverse backgrounds and understanding levels of users. With this paper, we investigate if free-form conversations can enhance users' comprehension of static explanations, improve acceptance and trust in the explanation methods, and facilitate human-AI collaboration. Participants are presented with static explanations, followed by a conversation with a human expert regarding the explanations. We measure the effect of the conversation on participants' ability to choose, from three machine learning models, the most accurate one based on explanations and their self-reported comprehension, acceptance, and trust. Empirical results show that conversations significantly improve comprehension, acceptance, trust, and collaboration. Our findings highlight the importance of customized model explanations in the format of free-form conversations and provide insights for the future design of conversational explanations.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 81 of its 170 references that are in the map. First posted on arXiv 25 Sep 2023.

Builds on

81 of 170 references.

References · 81

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  8. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  9. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  10. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  11. 2023A Survey of Large Language Modelscited by 2,576
  12. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
38 more
  1. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  2. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  3. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  4. 2018Sanity Checks for Saliency Mapscited by 1,156
  5. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  6. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  7. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  8. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 614
  9. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  10. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 399
  11. 2017The (Un)reliability of saliency methodscited by 333
  12. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 294
  13. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  14. 2018Metrics for Explainable AI: Challenges and Prospectscited by 260
  15. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  16. 2023MultiModal-GPT: A Vision and Language Model for Dialogue with Humanscited by 255
  17. 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 253
  18. 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
  19. 2019FACE: Feasible and Actionable Counterfactual Explanationscited by 233
  20. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  21. 2019Model-Agnostic Counterfactual Explanations for Consequential Decisionscited by 204
  22. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  23. 2018Abduction-Based Explanations for Machine Learning Modelscited by 160
  24. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
  25. 2020Debugging Tests for Model Explanationscited by 147
  26. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 142
  27. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  28. 2018A Symbolic Approach to Explaining Bayesian Network Classifierscited by 131
  29. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  30. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  31. 2021Human-Centered Explainable AI (XAI): From Algorithms to User Experiencescited by 123
  32. 2019Data Valuation using Reinforcement Learningcited by 116
  33. 2021The effectiveness of feature attribution methods and its correlation with automatic evaluation scorescited by 88
  34. 2017Frames: A Corpus for Adding Memory to Goal-Oriented Dialogue Systemscited by 85
  35. 2021A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutionscited by 78
  36. 2022Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderationcited by 67
  37. 2023Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Makingcited by 67
  38. 2022TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversationscited by 66

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 2.

5 papers.

  1. 2024Gradient based Feature Attribution in Explainable AI: A Technical Reviewcited by 34
  2. 2024Allowing humans to interactively guide machines where to look does not always improve human-AI team's classification accuracycited by 5
  3. 2025Conversational Explanations: Discussing Explainable AI with Non-AI Expertscited by 1
  4. 2026Exploring Plan Space through Conversation: An Agentic Framework for LLM-Mediated Explanations in Planningcited by 0
  5. 2026Towards Faithful Agentic XAI: A Verification Method and an Open-World Benchmark for Better Model Faithfulnesscited by 0