# TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations

arXiv: https://arxiv.org/abs/2207.04154 · HTML: https://mlsanity.com/paper/2207.04154/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 110 references that are in the map.
- First posted on arXiv 8 Jul 2022.

## Builds on (most cited first, 30 of 55)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [Constrained Language Models Yield Few-Shot Semantic Parsers](https://mlsanity.com/paper/2104.08768/) (2021)
- [A Survey on Dialogue Systems: Recent Advances and New Frontiers](https://mlsanity.com/paper/1711.01731/) (2017)
- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [Reliable Post hoc Explanations: Modeling Uncertainty in Explainability](https://mlsanity.com/paper/2008.05030/) (2020)
- [The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models](https://mlsanity.com/paper/2008.05122/) (2020)

## Built on by (most cited first, 30 of 66)

- [Generative AI](https://mlsanity.com/paper/2309.07930/) (2023)
- [Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant](https://mlsanity.com/paper/2502.01390/) (2025)
- [SHAP-IQ: Unified Approximation of any-order Shapley Interactions](https://mlsanity.com/paper/2303.01179/) (2023)
- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction](https://mlsanity.com/paper/2409.00265/) (2024)
- [LLMs for Explainable AI: A Comprehensive Survey](https://mlsanity.com/paper/2504.00125/) (2025)
- [Diagnosing Infeasible Optimization Problems Using Large Language Models](https://mlsanity.com/paper/2308.12923/) (2023)
- [Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant](https://mlsanity.com/paper/2501.17546/) (2025)
- [Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda](https://mlsanity.com/paper/2404.09554/) (2024)
- [Tell Me a Story! Narrative-Driven XAI with Large Language Models](https://mlsanity.com/paper/2309.17057/) (2023)
- [LLMs for XAI: Future Directions for Explaining Explanations](https://mlsanity.com/paper/2405.06064/) (2024)
- [LLMCheckup: Conversational Examination of Large Language Models via Interpretability Tools and Self-Explanations](https://mlsanity.com/paper/2401.12576/) (2024)
- [Building Trustworthy NeuroSymbolic AI Systems: Consistency, Reliability, Explainability, and Safety](https://mlsanity.com/paper/2312.06798/) (2023)
- [EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations](https://mlsanity.com/paper/2402.00491/) (2024)
- [Contrastive Explanations That Anticipate Human Misconceptions Can Improve Human Decision-Making Skills](https://mlsanity.com/paper/2410.04253/) (2024)
- [InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations](https://mlsanity.com/paper/2310.05592/) (2023)
- [Can LLMs Explain Themselves Counterfactually?](https://mlsanity.com/paper/2502.18156/) (2025)
- [Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety](https://mlsanity.com/paper/2506.05451/) (2025)
- [From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis](https://mlsanity.com/paper/2502.11919/) (2025)
- [Towards Unifying Evaluation of Counterfactual Explanations: Leveraging Large Language Models for Human-Centric Assessments](https://mlsanity.com/paper/2410.21131/) (2024)
- [OptiChat: Bridging Optimization Models and Practitioners with Large Language Models](https://mlsanity.com/paper/2501.08406/) (2025)
- [Measuring User Understanding in Dialogue-based XAI Systems](https://mlsanity.com/paper/2408.06960/) (2024)
- [Explingo: Explaining AI Predictions using Large Language Models](https://mlsanity.com/paper/2412.05145/) (2024)
- [AI Agent Behavioral Science](https://mlsanity.com/paper/2506.06366/) (2025)
- [May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainability](https://mlsanity.com/paper/2309.13965/) (2023)
- [Medical Foundation Models are Susceptible to Targeted Misinformation Attacks](https://mlsanity.com/paper/2309.17007/) (2023)
- [Allowing humans to interactively guide machines where to look does not always improve human-AI team's classification accuracy](https://mlsanity.com/paper/2404.05238/) (2024)
- [CoXQL: A Dataset for Parsing Explanation Requests in Conversational XAI Systems](https://mlsanity.com/paper/2406.08101/) (2024)
- [Explanatory Debiasing: Involving Domain Experts in the Data Generation Process to Mitigate Representation Bias in AI Systems](https://mlsanity.com/paper/2501.01441/) (2024)
- [QuAnTS: Question Answering on Time Series](https://mlsanity.com/paper/2511.05124/) (2025)

## Abstract (arXiv)

Machine Learning (ML) models are increasingly used to make critical decisions in real-world applications, yet they have become more complex, making them harder to understand. To this end, researchers have proposed several techniques to explain model predictions. However, practitioners struggle to use these explainability techniques because they often do not know which one to choose and how to interpret the results of the explanations. In this work, we address these challenges by introducing TalkToModel: an interactive dialogue system for explaining machine learning models through conversations. Specifically, TalkToModel comprises of three key components: 1) a natural language interface for engaging in conversations, making ML model explainability highly accessible, 2) a dialogue engine that adapts to any tabular model and dataset, interprets natural language, maps it to appropriate explanations, and generates text responses, and 3) an execution component that constructs the explanations. We carried out extensive quantitative and human subject evaluations of TalkToModel. Overall, we found the conversational system understands user inputs on novel datasets and models with high accuracy, demonstrating the system's capacity to generalize to new situations. In real-world evaluations with humans, 73% of healthcare workers (e.g., doctors and nurses) agreed they would use TalkToModel over baseline point-and-click systems for explainability in a disease prediction task, and 85% of ML professionals agreed TalkToModel was easier to use for computing explanations. Our findings demonstrate that TalkToModel is more effective for model explainability than existing systems, introducing a new category of explainability tools for practitioners. Code & demo released here: https://github.com/dylan-slack/TalkToModel.