mlsanityData to 7 Oct 2026

Mediators: Conversational Agents Explaining NLP Model Behavior

arXiv 2206.06029 ↗First posted 13 Jun 2022cs.CL cs.AI cs.HCcited by 7

Not on the map. Why?

Abstract (arXiv)

The human-centric explainable artificial intelligence (HCXAI) community has raised the need for framing the explanation process as a conversation between human and machine. In this position paper, we establish desiderata for Mediators, text-based conversational agents which are capable of explaining the behavior of neural models interactively using natural language. From the perspective of natural language processing (NLP) research, we engineer a blueprint of such a Mediator for the task of sentiment analysis and assess how far along current research is on the path towards dialogue-based explanations.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 63 of its 109 references that are in the map. First posted on arXiv 13 Jun 2022.

Builds on

63 of 109 references.

References · 63

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  3. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  4. 2022Red Teaming Language Models with Language Modelscited by 983
  5. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  6. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  7. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  8. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  9. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  10. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
  11. 2018QuAC : Question Answering in Contextcited by 379
  12. 2016Rationalizing Neural Predictionscited by 281
38 more
  1. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  2. 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 189
  3. 2020Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AIcited by 166
  4. 2020Measuring Association Between Labels and Free-Text Rationalescited by 159
  5. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  6. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  7. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  8. 2021Human-Centered Explainable AI (XAI): From Algorithms to User Experiencescited by 123
  9. 2019Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approachescited by 101
  10. 2020Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?cited by 95
  11. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  12. 2022Interpreting Language Models with Contrastive Explanationscited by 91
  13. 2020Human-centered Explainable AI: Towards a Reflective Sociotechnical Approachcited by 90
  14. 2020doc2dial: A Goal-Oriented Document-Grounded Dialogue Datasetcited by 88
  15. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 86
  16. 2019Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptionscited by 81
  17. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  18. 2019AllenNLP Interpret: A Framework for Explaining Predictions of NLP Modelscited by 74
  19. 2018Interpretation of Natural Language Rules in Conversational Machine Readingcited by 72
  20. 2020One Explanation Does Not Fit All: The Promise of Interactive Explanations for Machine Learning Transparencycited by 64
  21. 2022Dialog Inpainting: Turning Documents into Dialogscited by 56
  22. 2017The Promise and Peril of Human Evaluation for Model Interpretabilitycited by 52
  23. 2022Rethinking Explainability as a Dialogue: A Practitioner's Perspectivecited by 52
  24. 2021The Who in XAI: How AI Background Shapes Perceptions of AI Explanationscited by 48
  25. 2022In-Context Learning for Few-Shot Dialogue State Trackingcited by 47
  26. 2018The Challenge of Crafting Intelligible Intelligencecited by 46
  27. 2018Analyzing and Characterizing User Intent in Information-seeking Conversationscited by 36
  28. 2020ALICE: Active Learning with Contrastive Natural Language Explanationscited by 36
  29. 2021Question-Driven Design Process for Explainable AI User Experiencescited by 31
  30. 2021Refining Language Models with Compositional Explanationscited by 27
  31. 2019Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Datasetcited by 26
  32. 2021Rationales for Sequential Predictionscited by 26
  33. 2021Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanationscited by 26
  34. 2021What to Learn, and How: Toward Effective Learning from Rationalescited by 25
  35. 2022Human Interpretation of Saliency-based Explanation Over Textcited by 25
  36. 2019A Grounded Interaction Protocol for Explainable Artificial Intelligencecited by 22
  37. 2021Levels of explainable artificial intelligence for human-aligned conversational explanationscited by 21
  38. 2020Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experiencecited by 20

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 4; 2024: 1.

7 papers.

  1. 2022TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversationscited by 65
  2. 2023Diagnosing Infeasible Optimization Problems Using Large Language Modelscited by 27
  3. 2022Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methodscited by 13
  4. 2023InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanationscited by 11
  5. 2024JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Modelscited by 7
  6. 2023May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainabilitycited by 5
  7. 2023CommonsenseVIS: Visualizing and Understanding Commonsense Reasoning Capabilities of Natural Language Modelscited by 2