mlsanityData to 14 Oct 2026

Explainable Reinforcement Learning Through a Causal Lens

arXiv 1905.10958 ↗First posted 27 May 2019cs.LG cs.AI cs.HCcited by 177

Not on the map. Why?

Cited by 29 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Prevalent theories in cognitive science propose that humans understand and represent the knowledge of the world through causal relationships. In making sense of the world, we build causal models in our mind to encode cause-effect relations of events and use these to explain why new events happen. In this paper, we use causal models to derive causal explanations of behaviour of reinforcement learning agents. We present an approach that learns a structural causal model during reinforcement learning and encodes causal relationships between variables of interest. This model is then used to generate explanations of behaviour based on counterfactual analysis of the causal model. We report on a study with 120 participants who observe agents playing a real-time strategy game (Starcraft II) and then receive explanations of the agents' behaviour. We investigated: 1) participants' understanding gained by explanations through task prediction; 2) explanation satisfaction and 3) trust. Our results show that causal model explanations perform better on these measures compared to two other baseline explanation models.

Key facts. Cited by 177 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 37; 2022: 40; 2023: 36; 2024: 24; 2025: 30; 2026: 10.

177 papers.

Sort:

By area:

  1. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  2. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  3. 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
  4. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  5. 2021A Survey on Interpretable Reinforcement Learningcited by 73
  6. 2023A Comprehensive Survey on Deep Graph Representation Learningcited by 72
  7. 2023Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision supportcited by 66
  8. 2022Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learningcited by 53
  9. 2023Adversarial attacks and defenses in explainable artificial intelligence: A surveycited by 48
  10. 2023Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstractioncited by 46
  11. 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
  12. 2023A Survey on Causal Reinforcement Learningcited by 45
38 more
  1. 2021Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabicited by 40
  2. 2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 39
  3. 2021Counterfactual Explanations in Sequential Decision Making Under Uncertaintycited by 37
  4. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 34
  5. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  6. 2021Iterative Bounding MDPs: Learning Interpretable Policies via Non-Interpretable Methodscited by 31
  7. 2022A Survey of Explainable Reinforcement Learningcited by 31
  8. 2022Counterfactual harmcited by 29
  9. 2021Collective eXplainable AI: Explaining Cooperative Strategies and Agent Contribution in Multiagent Reinforcement Learning with Shapley Valuescited by 25
  10. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  11. 2023Formally Explaining Neural Networks within Reactive Systemscited by 24
  12. 2023Causal Reinforcement Learning: A Surveycited by 23
  13. 2023Explainable Reinforcement Learning via a Causal World Modelcited by 22
  14. 2021Levels of explainable artificial intelligence for human-aligned conversational explanationscited by 21
  15. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 21
  16. 2023Automated Cyber Defence: A Reviewcited by 21
  17. 2021Dynamic Causal Bayesian Optimizationcited by 17
  18. 2023State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understandingcited by 17
  19. 2021Contrastive Explanations of Plans Through Model Restrictionscited by 16
  20. 2021An Objective Metric for Explainable AI: How and Why to Estimate the Degree of Explainabilitycited by 16
  21. 2021A Survey on AI Assurancecited by 16
  22. 2021Inherently Explainable Reinforcement Learning in Natural Languagecited by 16
  23. 2023Causal Reasoning: Charting a Revolutionary Course for Next-Generation AI-Native Wireless Networkscited by 16
  24. 2021Explainable Autonomous Robots: A Survey and Perspectivecited by 15
  25. 2021Explainable Deep Learning in Healthcare: A Methodological Survey from an Attribution Viewcited by 15
  26. 2021Directive Explanations for Actionable Explainability in Machine Learning Applicationscited by 14
  27. 2022Survey on Fair Reinforcement Learning: Theory and Practicecited by 14
  28. 2023Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomescited by 14
  29. 2022Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunitiescited by 13
  30. 2023Finding Counterfactually Optimal Action Sequences in Continuous State Spacescited by 13
  31. 2022Explainability in reinforcement learning: perspective and positioncited by 11
  32. 2024Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learningcited by 11
  33. 2021Discovering User-Interpretable Capabilities of Black-Box Planning Agentscited by 10
  34. 2023Causal Explanations for Sequential Decision-Making in Multi-Agent Systemscited by 10
  35. 2021Training a Resilient Q-Network against Observational Interferencecited by 9
  36. 2021Learning Symbolic Rules for Interpretable Deep Reinforcement Learningcited by 9
  37. 2021Let the CAT out of the bag: Contrastive Attributed explanations for Textcited by 9
  38. 2021Explaining Deep Reinforcement Learning Agents In The Atari Domain through a Surrogate Modelcited by 9

50 most cited of 177 All 177 (JSON)