mlsanityData to 14 Oct 2026

Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptions

arXiv 1901.03729 ↗First posted 11 Jan 2019cs.AI cs.HCcited by 81

Not on the map. Why?

Abstract (arXiv)

Automated rationale generation is an approach for real-time explanation generation whereby a computational model learns to translate an autonomous agent's internal state and action data representations into natural language. Training on human explanation data can enable agents to learn to generate human-like explanations for their behavior. In this paper, using the context of an agent that plays Frogger, we describe (a) how to collect a corpus of explanations, (b) how to train a neural rationale generator to produce different styles of rationales, and (c) how people perceive these rationales. We conducted two user studies. The first study establishes the plausibility of each type of generated rationale and situates their user perceptions along the dimensions of confidence, humanlike-ness, adequate justification, and understandability. The second study further explores user preferences between the generated rationales with regard to confidence in the autonomous agent, communicating failure and unexpected behavior. Overall, we find alignment between the intended differences in features of the generated rationales and the perceived differences by users. Moreover, context permitting, participants preferred detailed rationales to form a stable mental model of the agent's behavior.

Key facts. Cited by 81 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 22; 2022: 13; 2023: 16; 2024: 16; 2025: 12; 2026: 2.

81 papers.

Sort:

By area:

  1. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  2. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  3. 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
  4. 2021Human-Centered Explainable AI (XAI): From Algorithms to User Experiencescited by 123
  5. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 85
  6. 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73
  7. 2024DocFinQA: A Long-Context Financial Reasoning Datasetcited by 60
  8. 2021The Who in XAI: How AI Background Shapes Perceptions of AI Explanationscited by 48
  9. 2021SelfExplain: A Self-Explaining Architecture for Neural Text Classifierscited by 44
  10. 2021Explainability Pitfalls: Beyond Dark Patterns in Explainable AIcited by 36
  11. 2023Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAIcited by 33
  12. 2023A Systematic Literature Review of User Trust in AI-Enabled Systems: An HCI Perspectivecited by 33
38 more
  1. 2021Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recoverycited by 32
  2. 2021Local Interpretations for Explainable Natural Language Processing: A Surveycited by 31
  3. 2021On the Diversity and Limits of Human Explanationscited by 30
  4. 2023Selective Explanations: Leveraging Human Input to Align Explainable AIcited by 29
  5. 2021Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updatescited by 27
  6. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  7. 2024Properties and Challenges of LLM-Generated Explanationscited by 24
  8. 2021Levels of explainable artificial intelligence for human-aligned conversational explanationscited by 21
  9. 2022Explainable Artificial Intelligence (XAI) from a user perspective- A synthesis of prior literature and problematizing avenues for future researchcited by 20
  10. 2025Citations and Trust in LLM Generated Responsescited by 18
  11. 2023State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understandingcited by 17
  12. 2021Inherently Explainable Reinforcement Learning in Natural Languagecited by 16
  13. 2021Explainable Autonomous Robots: A Survey and Perspectivecited by 15
  14. 2021Directive Explanations for Actionable Explainability in Machine Learning Applicationscited by 14
  15. 2021Explainable Activity Recognition for Smart Home Systemscited by 14
  16. 2021Semantic-Based Explainable AI: Leveraging Semantic Scene Graphs and Pairwise Ranking to Explain Robot Failurescited by 13
  17. 2022Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methodscited by 13
  18. 2023ALMANACS: A Simulatability Benchmark for Language Model Explainabilitycited by 11
  19. 2022Trust and Reliance in XAI -- Distinguishing Between Attitudinal and Behavioral Measurescited by 10
  20. 2022The Algorithmic Imprintcited by 10
  21. 2021Explaining Deep Reinforcement Learning Agents In The Atari Domain through a Surrogate Modelcited by 9
  22. 2024Unearthing Skill-Level Insights for Understanding Trade-Offs of Foundation Modelscited by 9
  23. 2024Don't be Fooled: The Misinformation Effect of Explanations in Human-AI Collaborationcited by 8
  24. 2022Mediators: Conversational Agents Explaining NLP Model Behaviorcited by 7
  25. 2022Subgoal-Based Explanations for Unreliable Intelligent Decision Support Systemscited by 6
  26. 2022Social Construction of XAI: Do We Need One Definition to Rule Them All?cited by 6
  27. 2025Translate Policy to Language: Flow Matching Generated Rewards for LLM Explanationscited by 6
  28. 2022Explainable Knowledge Graph Embedding: Inference Reconciliation for Knowledge Inferences Supporting Robot Actionscited by 5
  29. 2022Experiential Explanations for Reinforcement Learningcited by 5
  30. 2023Users are the North Star for AI Transparencycited by 5
  31. 2025Standardizing Intelligence: Aligning Generative AI for Regulatory and Operational Compliancecited by 5
  32. 2021Influencing Reinforcement Learning through Natural Language Guidancecited by 4
  33. 2023Towards the design of user-centric strategy recommendation systems for collaborative Human-AI taskscited by 4
  34. 2023Diagrammatization and Abduction to Improve AI Interpretability With Domain-Aligned Explanations for Medical Diagnosiscited by 4
  35. 2024Free-text Rationale Generation under Readability Level Controlcited by 4
  36. 2025Model-Agnostic Policy Explanations with Large Language Modelscited by 4
  37. 2022A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challengescited by 3
  38. 2023Advantage Actor-Critic with Reasoner: Explaining the Agent's Behavior from an Exploratory Perspectivecited by 3

50 most cited of 81 All 81 (JSON)