# Introspection-based Explainable Reinforcement Learning in Episodic and Non-episodic Scenarios

arXiv: https://arxiv.org/abs/2211.12930 · HTML: https://mlsanity.com/paper/2211.12930/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 37 references that are in the map.
- First posted on arXiv 23 Nov 2022.

## Builds on (most cited first, 14 of 14)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Programmatically Interpretable Reinforcement Learning](https://mlsanity.com/paper/1804.02477/) (2018)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning](https://mlsanity.com/paper/1712.07294/) (2017)
- [Interpretable Policies for Reinforcement Learning by Genetic Programming](https://mlsanity.com/paper/1712.04170/) (2017)
- [Explainability in Human-Agent Systems](https://mlsanity.com/paper/1904.08123/) (2019)
- [Explainable Goal-Driven Agents and Robots -- A Comprehensive Review](https://mlsanity.com/paper/2004.09705/) (2020)
- [Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey](https://mlsanity.com/paper/2108.09003/) (2021)
- [Explainable robotic systems: Understanding goal-driven actions in a reinforcement learning scenario](https://mlsanity.com/paper/2006.13615/) (2020)
- [Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1904.01318/) (2019)
- [A Comparison of Humanoid Robot Simulators: A Quantitative Approach](https://mlsanity.com/paper/2008.04627/) (2020)
- [Evaluating Human-like Explanations for Robot Actions in Reinforcement Learning Scenarios](https://mlsanity.com/paper/2207.03214/) (2022)
- [Explainable Deep Reinforcement Learning Using Introspection in a Non-episodic Task](https://mlsanity.com/paper/2108.08911/) (2021)

## Abstract (arXiv)

With the increasing presence of robotic systems and human-robot environments in today's society, understanding the reasoning behind actions taken by a robot is becoming more important. To increase this understanding, users are provided with explanations as to why a specific action was taken. Among other effects, these explanations improve the trust of users in their robotic partners. One option for creating these explanations is an introspection-based approach which can be used in conjunction with reinforcement learning agents to provide probabilities of success. These can in turn be used to reason about the actions taken by the agent in a human-understandable fashion. In this work, this introspection-based approach is developed and evaluated further on the basis of an episodic and a non-episodic robotics simulation task. Furthermore, an additional normalization step to the Q-values is proposed, which enables the usage of the introspection-based approach on negative and comparatively small Q-values. Results obtained show the viability of introspection for episodic robotics tasks and, additionally, that the introspection-based approach can be used to generate explanations for the actions taken in a non-episodic robotics environment as well.