# Explainable Artificial Intelligence (XAI) for Increasing User Trust in Deep Reinforcement Learning Driven Autonomous Systems

arXiv: https://arxiv.org/abs/2106.03775 · HTML: https://mlsanity.com/paper/2106.03775/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 17 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Distance-based Confidence Score for Neural Network Classifiers](https://mlsanity.com/paper/1709.09844/) (2017)
- [Measuring and Characterizing Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/1812.02868/) (2018)
- [Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning](https://mlsanity.com/paper/1905.02825/) (2019)

## Built on by (most cited first, 7 of 7)

- [Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learning](https://mlsanity.com/paper/2208.07811/) (2022)
- [Explainable Artificial Intelligence (XAI) for Internet of Things: A Survey](https://mlsanity.com/paper/2206.04800/) (2022)
- [Explainability in Deep Reinforcement Learning, a Review into Current Methods and Applications](https://mlsanity.com/paper/2207.01911/) (2022)
- [A Comprehensive Survey and Classification of Evaluation Criteria for Trustworthy Artificial Intelligence](https://mlsanity.com/paper/2410.17281/) (2024)
- [Explainable Interface for Human-Autonomy Teaming: A Survey](https://mlsanity.com/paper/2405.02583/) (2024)
- [Robust Optimization for Mitigating Reward Hacking with Correlated Proxies](https://mlsanity.com/paper/2604.12086/) (2026)
- [Measuring Interventional Robustness in Reinforcement Learning](https://mlsanity.com/paper/2209.09058/) (2022)

## Abstract (arXiv)

We consider the problem of providing users of deep Reinforcement Learning (RL) based systems with a better understanding of when their output can be trusted. We offer an explainable artificial intelligence (XAI) framework that provides a three-fold explanation: a graphical depiction of the systems generalization and performance in the current game state, how well the agent would play in semantically similar environments, and a narrative explanation of what the graphical information implies. We created a user-interface for our XAI framework and evaluated its efficacy via a human-user experiment. The results demonstrate a statistically significant increase in user trust and acceptance of the AI system with explanation, versus the AI system without explanation.