# Counterfactual States for Atari Agents via Generative Deep Learning

arXiv: https://arxiv.org/abs/1909.12969 · HTML: https://mlsanity.com/paper/1909.12969/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2019.

## Built on by (most cited first, 14 of 14)

- [What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research](https://mlsanity.com/paper/2102.07817/) (2021)
- [If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques](https://mlsanity.com/paper/2103.01035/) (2021)
- [Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero](https://mlsanity.com/paper/2310.16410/) (2023)
- [Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learning](https://mlsanity.com/paper/2101.12446/) (2021)
- [Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptation](https://mlsanity.com/paper/2307.06333/) (2023)
- [Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunities](https://mlsanity.com/paper/2210.11846/) (2022)
- [Explaining Reinforcement Learning Policies through Counterfactual Trajectories](https://mlsanity.com/paper/2201.12462/) (2022)
- [RACCER: Towards Reachable and Certain Counterfactual Explanations for Reinforcement Learning](https://mlsanity.com/paper/2303.04475/) (2023)
- [ReCCoVER: Detecting Causal Confusion for Explainable Reinforcement Learning](https://mlsanity.com/paper/2203.11211/) (2022)
- [Identifying Reasoning Flaws in Planning-Based RL Using Tree Explanations](https://mlsanity.com/paper/2109.13978/) (2021)
- [Causal Robot Communication Inspired by Observational Learning Insights](https://mlsanity.com/paper/2203.09114/) (2022)
- [Personalized Path Recourse for Reinforcement Learning Agents](https://mlsanity.com/paper/2312.08724/) (2023)
- [Backward explanations via redefinition of predicates](https://mlsanity.com/paper/2408.02606/) (2024)
- [Explaining Reinforcement Learning Decisions in Self-adaptive Systems](https://mlsanity.com/paper/2608.14620/) (2026)

## Abstract (arXiv)

Although deep reinforcement learning agents have produced impressive results in many domains, their decision making is difficult to explain to humans. To address this problem, past work has mainly focused on explaining why an action was chosen in a given state. A different type of explanation that is useful is a counterfactual, which deals with "what if?" scenarios. In this work, we introduce the concept of a counterfactual state to help humans gain a better understanding of what would need to change (minimally) in an Atari game image for the agent to choose a different action. We introduce a novel method to create counterfactual states from a generative deep learning architecture. In addition, we evaluate the effectiveness of counterfactual states on human participants who are not machine learning experts. Our user study results suggest that our generated counterfactual states are useful in helping non-expert participants gain a better understanding of an agent's decision making process.