# Contrastive Explanations for Reinforcement Learning via Embedded Self Predictions

arXiv: https://arxiv.org/abs/2010.05180 · HTML: https://mlsanity.com/paper/2010.05180/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2020.

## Built on by (most cited first, 13 of 13)

- [A Survey of Explainable Reinforcement Learning](https://mlsanity.com/paper/2202.08434/) (2022)
- [Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey](https://mlsanity.com/paper/2108.09003/) (2021)
- [Concept Learning for Interpretable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.12232/) (2023)
- [Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomes](https://mlsanity.com/paper/2312.11118/) (2023)
- [Explainable Reinforcement Learning via Model Transforms](https://mlsanity.com/paper/2209.12006/) (2022)
- [Explainable Multi-Agent Reinforcement Learning for Temporal Queries](https://mlsanity.com/paper/2305.10378/) (2023)
- [TalkToAgent: A Human-centric Explanation of Reinforcement Learning Agents with Large Language Models](https://mlsanity.com/paper/2509.04809/) (2025)
- [(When) Are Contrastive Explanations of Reinforcement Learning Helpful?](https://mlsanity.com/paper/2211.07719/) (2022)
- [Discovering Object-Centric Generalized Value Functions From Pixels](https://mlsanity.com/paper/2304.13892/) (2023)
- [Explainable reinforcement learning from human feedback to improve alignment](https://mlsanity.com/paper/2512.13837/) (2025)
- [Beyond Value: CHECKLIST for Testing Inferences in Planning-Based RL](https://mlsanity.com/paper/2206.02039/) (2022)
- [How to Measure Human-AI Prediction Accuracy in Explainable AI Systems](https://mlsanity.com/paper/2409.00069/) (2024)
- [Pay Attention to What and Where? Interpretable Feature Extractor in Vision-based Deep Reinforcement Learning](https://mlsanity.com/paper/2504.10071/) (2025)

## Abstract (arXiv)

We investigate a deep reinforcement learning (RL) architecture that supports explaining why a learned agent prefers one action over another. The key idea is to learn action-values that are directly represented via human-understandable properties of expected futures. This is realized via the embedded self-prediction (ESP)model, which learns said properties in terms of human provided features. Action preferences can then be explained by contrasting the future properties predicted for each action. To address cases where there are a large number of features, we develop a novel method for computing minimal sufficient explanations from anESP. Our case studies in three domains, including a complex strategy game, show that ESP models can be effectively learned and support insightful explanations.