# Contrastive Explanations for Comparing Preferences of Reinforcement Learning Agents

arXiv: https://arxiv.org/abs/2112.09462 · HTML: https://mlsanity.com/paper/2112.09462/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 19 references that are in the map.
- First posted on arXiv 17 Dec 2021.

## Builds on (most cited first, 11 of 11)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Techniques for Interpretable Machine Learning](https://mlsanity.com/paper/1808.00033/) (2018)
- [Explainable Reinforcement Learning Through a Causal Lens](https://mlsanity.com/paper/1905.10958/) (2019)
- [Visualizing and Understanding Atari Agents](https://mlsanity.com/paper/1711.00138/) (2017)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [Interestingness Elements for Explainable Reinforcement Learning: Understanding Agents' Capabilities and Limitations](https://mlsanity.com/paper/1912.09007/) (2019)
- [Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences](https://mlsanity.com/paper/1807.08706/) (2018)
- [Enabling Robots to Communicate their Objectives](https://mlsanity.com/paper/1702.03465/) (2017)
- [Distal Explanations for Model-free Explainable Reinforcement Learning](https://mlsanity.com/paper/2001.10284/) (2020)
- [Autonomous Self-Explanation of Behavior for Interactive Reinforcement Learning Agents](https://mlsanity.com/paper/1810.08811/) (2018)

## Built on by (most cited first, 5 of 5)

- [Explainability in Deep Reinforcement Learning, a Review into Current Methods and Applications](https://mlsanity.com/paper/2207.01911/) (2022)
- [ReCCoVER: Detecting Causal Confusion for Explainable Reinforcement Learning](https://mlsanity.com/paper/2203.11211/) (2022)
- [AutoDOViz: Human-Centered Automation for Decision Optimization](https://mlsanity.com/paper/2302.09688/) (2023)
- [Towards Regulatable AI Systems: Technical Gaps and Policy Opportunities](https://mlsanity.com/paper/2306.12609/) (2023)
- [Conveying Autonomous Robot Capabilities through Contrasting Behaviour Summaries](https://mlsanity.com/paper/2304.00367/) (2023)

## Abstract (arXiv)

In complex tasks where the reward function is not straightforward and consists of a set of objectives, multiple reinforcement learning (RL) policies that perform task adequately, but employ different strategies can be trained by adjusting the impact of individual objectives on reward function. Understanding the differences in strategies between policies is necessary to enable users to choose between offered policies, and can help developers understand different behaviors that emerge from various reward functions and training hyperparameters in RL systems. In this work we compare behavior of two policies trained on the same task, but with different preferences in objectives. We propose a method for distinguishing between differences in behavior that stem from different abilities from those that are a consequence of opposing preferences of two RL agents. Furthermore, we use only data on preference-based differences in order to generate contrasting explanations about agents' preferences. Finally, we test and evaluate our approach on an autonomous driving task and compare the behavior of a safety-oriented policy and one that prefers speed.