Counterfactual Explainer Framework for Deep Reinforcement Learning Models Using Policy Distillation
Not on the map. Why?
Abstract (arXiv)
Deep Reinforcement Learning (DRL) has demonstrated promising capability in solving complex control problems. However, DRL applications in safety-critical systems are hindered by the inherent lack of robust verification techniques to assure their performance in such applications. One of the key requirements of the verification process is the development of effective techniques to explain the system functionality, i.e., why the system produces specific results in given circumstances. Recently, interpretation methods based on the Counterfactual (CF) explanation approach have been proposed to address the problem of explanation in DRLs. This paper proposes a novel CF explanation framework to explain the decisions made by a black-box DRL. To evaluate the efficacy of the proposed explanation framework, we carried out several experiments in the domains of automated driving systems and Atari Pong game. Our analysis demonstrates that the proposed framework generates plausible and meaningful explanations for various decisions made by deep underlying DRLs. Source codes are available at: urlhttps://github.com/Amir-Samadi/Counterfactual-Explanation
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 44 of its 59 references that are in the map. First posted on arXiv 25 May 2023.
Builds on
44 of 59 references.
References · 44
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2013Intriguing properties of neural networkscited by 5,155
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
- 2016OpenAI Gymcited by 1,736
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
- 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
32 more
- 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,039
- 2016End to End Learning for Self-Driving Carscited by 1,009
- 2018Progress & Compress: A scalable framework for continual learningcited by 520
- 2013Do Deep Nets Really Need to be Deep?cited by 474
- 2015Policy Distillationcited by 440
- 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
- 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 404
- 2019Counterfactual Visual Explanationscited by 313
- 2017Distral: Robust Multitask Reinforcement Learningcited by 233
- 2018Local Rule-Based Explanations of Black Box Decision Systemscited by 166
- 2020Multi-Objective Counterfactual Explanationscited by 157
- 2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 134
- 2020A Survey of End-to-End Driving: Architectures and Training Methodscited by 123
- 2018Grounding Visual Explanationscited by 93
- 2020Transient Non-Stationarity and Generalisation in Deep Reinforcement Learningcited by 75
- 2019Grounding Human-to-Vehicle Advice for Self-driving Vehiclescited by 71
- 2017Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Carcited by 66
- 2021Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applicationscited by 59
- 2021Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanationscited by 56
- 2020SCOUT: Self-aware Discriminant Counterfactual Explanationscited by 52
- 2017Applications of Deep Learning and Reinforcement Learning to Biological Datacited by 45
- 2021STEEX: Steering Counterfactual Explanations with Semanticscited by 40
- 2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 39
- 2019DisCoRL: Continual Reinforcement Learning via Policy Distillationcited by 36
- 2019CERTIFAI: Counterfactual Explanations for Robustness, Transparency, Interpretability, and Fairness of Artificial Intelligence modelscited by 34
- 2019Why Does My Model Fail? Contrastive Local Explanations for Retail Forecastingcited by 31
- 2019Counterfactual Explanation Algorithms for Behavioral and Textual Datacited by 26
- 2021ReLAX: Reinforcement Learning Agent eXplainer for Arbitrary Predictive Modelscited by 25
- 2019Generating Counterfactual and Contrastive Explanations using SHAPcited by 24
- 2020Semantics and explanation: why counterfactual explanations produce adversarial examples in deep neural networkscited by 24
- 2023GANterfactual-RL: Understanding Reinforcement Learning Agents' Strategies through Visual Counterfactual Explanationscited by 17
- 2021Synthesising Reinforcement Learning Policies through Set-Valued Inductive Rule Learningcited by 2
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.