# Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences

arXiv: https://arxiv.org/abs/1807.08706 · HTML: https://mlsanity.com/paper/1807.08706/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2018.

## Built on by (most cited first, 30 of 44)

- [A Survey on Interpretable Reinforcement Learning](https://mlsanity.com/paper/2112.13112/) (2021)
- [Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learning](https://mlsanity.com/paper/2101.12446/) (2021)
- [Explainable Deep Reinforcement Learning: State of the Art and Challenges](https://mlsanity.com/paper/2301.09937/) (2023)
- [Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey](https://mlsanity.com/paper/2108.09003/) (2021)
- [Explainable Autonomous Robots: A Survey and Perspective](https://mlsanity.com/paper/2105.02658/) (2021)
- [A Human-Centered Safe Robot Reinforcement Learning Framework with Interactive Behaviors](https://mlsanity.com/paper/2302.13137/) (2023)
- [Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomes](https://mlsanity.com/paper/2312.11118/) (2023)
- [Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunities](https://mlsanity.com/paper/2210.11846/) (2022)
- [Explainability in reinforcement learning: perspective and position](https://mlsanity.com/paper/2203.11547/) (2022)
- [A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2310.06253/) (2023)
- [Feature-Based Interpretable Reinforcement Learning based on State-Transition Models](https://mlsanity.com/paper/2105.07099/) (2021)
- [Contrastive Explanations for Comparing Preferences of Reinforcement Learning Agents](https://mlsanity.com/paper/2112.09462/) (2021)
- [Experiential Explanations for Reinforcement Learning](https://mlsanity.com/paper/2210.04723/) (2022)
- [Natural Language Specification of Reinforcement Learning Policies through Differentiable Decision Trees](https://mlsanity.com/paper/2101.07140/) (2021)
- [ReCCoVER: Detecting Causal Confusion for Explainable Reinforcement Learning](https://mlsanity.com/paper/2203.11211/) (2022)
- [ProtoX: Explaining a Reinforcement Learning Agent via Prototyping](https://mlsanity.com/paper/2211.03162/) (2022)
- [A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges](https://mlsanity.com/paper/2211.06665/) (2022)
- [CODEX: A Cluster-Based Method for Explainable Reinforcement Learning](https://mlsanity.com/paper/2312.04216/) (2023)
- [Cultural Bias in Explainable AI Research: A Systematic Analysis](https://mlsanity.com/paper/2403.05579/) (2024)
- [TalkToAgent: A Human-centric Explanation of Reinforcement Learning Agents with Large Language Models](https://mlsanity.com/paper/2509.04809/) (2025)
- [Policy Optimization with Sparse Global Contrastive Explanations](https://mlsanity.com/paper/2207.06269/) (2022)
- [Reward Learning with Trees: Methods and Evaluation](https://mlsanity.com/paper/2210.01007/) (2022)
- [Abstracted Trajectory Visualization for Explainability in Reinforcement Learning](https://mlsanity.com/paper/2402.07928/) (2024)
- [One-shot Policy Elicitation via Semantic Reward Manipulation](https://mlsanity.com/paper/2101.01860/) (2021)
- [(When) Are Contrastive Explanations of Reinforcement Learning Helpful?](https://mlsanity.com/paper/2211.07719/) (2022)
- [Contrastive Explainable Clustering with Differential Privacy](https://mlsanity.com/paper/2406.04610/) (2024)
- [Explainable Reinforcement Learning Agents Using World Models](https://mlsanity.com/paper/2505.08073/) (2025)
- [System-Agnostic Meta-Learning for MDP-based Dynamic Scheduling via Descriptive Policy](https://mlsanity.com/paper/2201.07051/) (2022)
- [Beyond Value: CHECKLIST for Testing Inferences in Planning-Based RL](https://mlsanity.com/paper/2206.02039/) (2022)
- [Measuring Interventional Robustness in Reinforcement Learning](https://mlsanity.com/paper/2209.09058/) (2022)

## Abstract (arXiv)

Machine Learning models become increasingly proficient in complex tasks. However, even for experts in the field, it can be difficult to understand what the model learned. This hampers trust and acceptance, and it obstructs the possibility to correct the model. There is therefore a need for transparency of machine learning models. The development of transparent classification models has received much attention, but there are few developments for achieving transparent Reinforcement Learning (RL) models. In this study we propose a method that enables a RL agent to explain its behavior in terms of the expected consequences of state transitions and outcomes. First, we define a translation of states and actions to a description that is easier to understand for human users. Second, we developed a procedure that enables the agent to obtain the consequences of a single action, as well as its entire policy. The method calculates contrasts between the consequences of a policy derived from a user query, and of the learned policy of the agent. Third, a format for generating explanations was constructed. A pilot survey study was conducted to explore preferences of users for different explanation properties. Results indicate that human users tend to favor explanations about policy rather than about single actions.