mlsanityData to 14 Oct 2026

ASQ-IT: Interactive Explanations for Reinforcement-Learning Agents

arXiv 2301.09941 ↗First posted 24 Jan 2023cs.AI cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

As reinforcement learning methods increasingly amass accomplishments, the need for comprehending their solutions becomes more crucial. Most explainable reinforcement learning (XRL) methods generate a static explanation depicting their developers' intuition of what should be explained and how. In contrast, literature from the social sciences proposes that meaningful explanations are structured as a dialog between the explainer and the explainee, suggesting a more active role for the user and her communication with the agent. In this paper, we present ASQ-IT -- an interactive tool that presents video clips of the agent acting in its environment based on queries given by the user that describe temporal properties of behaviors of interest. Our approach is based on formal methods: queries in ASQ-IT's user interface map to a fragment of Linear Temporal Logic over finite traces (LTLf), which we developed, and our algorithm for query processing is based on automata theory. User studies show that end-users can understand and formulate queries in ASQ-IT, and that using ASQ-IT assists users in identifying faulty agent behaviors.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 88 references that are in the map. First posted on arXiv 24 Jan 2023.

Builds on

33 of 88 references.

References · 33

  1. 2017Deep reinforcement learning from human preferencescited by 3,912
  2. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  3. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  4. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  5. 2018Metrics for Explainable AI: Challenges and Prospectscited by 260
  6. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  7. 2020Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Reviewcited by 174
  8. 2017Visualizing and Understanding Atari Agentscited by 141
  9. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  10. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  11. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  12. 2015The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classificationcited by 76
21 more
  1. 2019An Evaluation of the Human-Interpretability of Explanationcited by 71
  2. 2018Toward Interpretable Deep Reinforcement Learning with Linear Model U-Treescited by 65
  3. 2019Interestingness Elements for Explainable Reinforcement Learning: Understanding Agents' Capabilities and Limitationscited by 50
  4. 2018Establishing Appropriate Trust via Critical Statescited by 48
  5. 2022Towards Formal XAI: Formally Approximate Minimal Explanations of Neural Networkscited by 42
  6. 2020Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitionscited by 41
  7. 2021A Survey on Interactive Reinforcement Learning: Design Principles and Open Challengescited by 41
  8. 2019Generation of Policy-Level Explanations for Reinforcement Learningcited by 37
  9. 2017Enabling Robots to Communicate their Objectivescited by 35
  10. 2019Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulationcited by 35
  11. 2020Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Mapscited by 30
  12. 2019Exploring Computational User Models for Agent Policy Summarizationcited by 29
  13. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  14. 2016Particle Swarm Optimization for Generating Interpretable Fuzzy Reinforcement Learning Policiescited by 19
  15. 2019Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agentscited by 15
  16. 2021From Verification to Causality-based Explicationscited by 12
  17. 2020Formal Methods with a Touch of Magiccited by 10
  18. 2022Integrating Policy Summaries with Reward Decomposition for Explaining Reinforcement Learning Agentscited by 10
  19. 2021Interactive Explanations: Diagnosis and Repair of Reinforcement Learning Based Agent Behaviorscited by 8
  20. 2022Explaining Hyperproperty Violationscited by 3
  21. 2020Temporal-Logic Query Checking over Finite Data Streamscited by 2

Built on by

Not cited yet.