mlsanityData to 7 Oct 2026

IxDRL: A Novel Explainable Deep Reinforcement Learning Toolkit based on Analyses of Interestingness

arXiv 2307.08933 ↗First posted 18 Jul 2023cs.AI cs.HC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In recent years, advances in deep learning have resulted in a plethora of successes in the use of reinforcement learning (RL) to solve complex sequential decision tasks with high-dimensional inputs. However, existing systems lack the necessary mechanisms to provide humans with a holistic view of their competence, presenting an impediment to their adoption, particularly in critical applications where the decisions an agent makes can have significant consequences. Yet, existing RL-based systems are essentially competency-unaware in that they lack the necessary interpretation mechanisms to allow human operators to have an insightful, holistic view of their competency. Towards more explainable Deep RL (xDRL), we propose a new framework based on analyses of interestingness. Our tool provides various measures of RL agent competence stemming from interestingness analysis and is applicable to a wide range of RL algorithms, natively supporting the popular RLLib toolkit. We showcase the use of our framework by applying the proposed pipeline in a set of scenarios of varying complexity. We empirically assess the capability of the approach in identifying agent behavior patterns and competency-controlling conditions, and the task elements mostly responsible for an agent's competence, based on global and local analyses of interestingness. Overall, we show that our framework can provide agent designers with insights about RL agent competence, both their capabilities and limitations, enabling more informed decisions about interventions, additional training, and other interactions in collaborative human-machine settings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 55 references that are in the map. First posted on arXiv 18 Jul 2023.

Builds on

28 of 55 references.

References · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  6. 2015Prioritized Experience Replaycited by 1,070
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  8. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  9. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  10. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  11. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  12. 2019Self-Supervised Exploration via Disagreementcited by 263
16 more
  1. 2018Metrics for Explainable AI: Challenges and Prospectscited by 257
  2. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  3. 2017Visualizing and Understanding Atari Agentscited by 141
  4. 2020Explainable Reinforcement Learning: A Surveycited by 139
  5. 2018Model-Based Active Explorationcited by 105
  6. 2020Explainability in Deep Reinforcement Learningcited by 96
  7. 2016Graying the black box: Understanding DQNscited by 68
  8. 2019Interestingness Elements for Explainable Reinforcement Learning: Understanding Agents' Capabilities and Limitationscited by 50
  9. 2018Establishing Appropriate Trust via Critical Statescited by 47
  10. 2018Contrastive Explanations for Reinforcement Learning in terms of Expected Consequencescited by 44
  11. 2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 38
  12. 2018Learning Finite State Representations of Recurrent Policy Networkscited by 37
  13. 2017Enabling Robots to Communicate their Objectivescited by 34
  14. 2019Exploring Computational User Models for Agent Policy Summarizationcited by 29
  15. 2022Outcome-Guided Counterfactuals for Reinforcement Learning Agents from a Jointly Trained Generative Latent Spacecited by 2
  16. 2022A Framework for Understanding and Visualizing Strategies of RL Agentscited by 2

Built on by

Not cited yet.