Causal Explanation for Reinforcement Learning: Quantifying State and Temporal Importance
Not on the map. Why?
Abstract (arXiv)
Explainability plays an increasingly important role in machine learning. Furthermore, humans view the world through a causal lens and thus prefer causal explanations over associational ones. Therefore, in this paper, we develop a causal explanation mechanism that quantifies the causal importance of states on actions and such importance over time. We also demonstrate the advantages of our mechanism over state-of-the-art associational methods in terms of RL policy explanation through a series of simulation studies, including crop irrigation, Blackjack, collision avoidance, and lunar lander.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 24 Oct 2022.
Builds on
24 of 41 references.
References · 24
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
- 2016OpenAI Gymcited by 1,736
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
- 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
- 2013Causal Discovery with Continuous Additive Noise Modelscited by 255
- 2018Programmatically Interpretable Reinforcement Learningcited by 193
- 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
- 2017Visualizing and Understanding Atari Agentscited by 141
12 more
- 2020Explainable Reinforcement Learning: A Surveycited by 141
- 2019CXPlain: Causal Explanations for Model Interpretation under Uncertaintycited by 98
- 2020Explainability in Deep Reinforcement Learningcited by 97
- 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
- 2016Discovering Causal Signals in Imagescited by 79
- 2019Neural Network Attributions: A Causal Perspectivecited by 77
- 2021gCastle: A Python Toolbox for Causal Discoverycited by 61
- 2019Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attributioncited by 50
- 2019Causal Discovery Toolbox: Uncover causal relationships in Pythoncited by 49
- 2018Transparency and Explanation in Deep Reinforcement Learning Neural Networkscited by 40
- 2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 39
- 2017Bidirectional Conditional Generative Adversarial Networkscited by 10
Built on by
Citing papers per year: 2023: 1.
1 paper.