# Learning Nonlinear Causal Reductions to Explain Reinforcement Learning Policies

arXiv: https://arxiv.org/abs/2507.14901 · HTML: https://mlsanity.com/paper/2507.14901/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 20 Jul 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 26 of 26)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Reinforcement learning based recommender systems: A survey](https://mlsanity.com/paper/2101.06286/) (2021)
- [Explainable Reinforcement Learning Through a Causal Lens](https://mlsanity.com/paper/1905.10958/) (2019)
- [Visualizing and Understanding Atari Agents](https://mlsanity.com/paper/1711.00138/) (2017)
- [Foundations of Structural Causal Models with Cycles and Latent Variables](https://mlsanity.com/paper/1611.06221/) (2016)
- [Causal Consistency of Structural Equation Models](https://mlsanity.com/paper/1707.00819/) (2017)
- [Abstracting Causal Models](https://mlsanity.com/paper/1812.03789/) (2018)
- [Towards Interpretable Reinforcement Learning Using Attention Augmented Agents](https://mlsanity.com/paper/1906.02500/) (2019)
- [Graying the black box: Understanding DQNs](https://mlsanity.com/paper/1602.02658/) (2016)
- [Approximate Causal Abstraction](https://mlsanity.com/paper/1906.11583/) (2019)
- [Visual Causal Feature Learning](https://mlsanity.com/paper/1412.2309/) (2014)
- [Transparency and Explanation in Deep Reinforcement Learning Neural Networks](https://mlsanity.com/paper/1809.06061/) (2018)
- [Generation of Policy-Level Explanations for Reinforcement Learning](https://mlsanity.com/paper/1905.12044/) (2019)
- [Learning to Play Table Tennis From Scratch using Muscular Robots](https://mlsanity.com/paper/2006.05935/) (2020)
- [Compositional Abstraction Error and a Category of Causal Models](https://mlsanity.com/paper/2103.15758/) (2021)
- [Causal Abstraction with Soft Interventions](https://mlsanity.com/paper/2211.12270/) (2022)
- [Explaining Reinforcement Learning to Mere Mortals: An Empirical Study](https://mlsanity.com/paper/1903.09708/) (2019)
- [Explainable robotic systems: Understanding goal-driven actions in a reinforcement learning scenario](https://mlsanity.com/paper/2006.13615/) (2020)
- [Homomorphism Autoencoder -- Learning Group Structured Representations from Observed Transitions](https://mlsanity.com/paper/2207.12067/) (2022)
- [Targeted Reduction of Causal Models](https://mlsanity.com/paper/2311.18639/) (2023)
- [Safe & Accurate at Speed with Tendons: A Robot Arm for Exploring Dynamic Motion](https://mlsanity.com/paper/2307.02654/) (2023)

## Built on by (most cited first, 1 of 1)

- [Coarsening Causal DAG Models](https://mlsanity.com/paper/2601.10531/) (2026)

## Abstract (arXiv)

Why do reinforcement learning (RL) policies fail or succeed? This is a challenging question due to the complex, high-dimensional nature of agent-environment interactions. In this work, we take a causal perspective on explaining the behavior of RL policies by viewing the states, actions, and rewards as variables in a low-level causal model. We introduce random perturbations to policy actions during execution and observe their effects on the cumulative reward, learning a simplified high-level causal model that explains these relationships. To this end, we develop a nonlinear Causal Model Reduction framework that ensures approximate interventional consistency, meaning the simplified high-level model responds to interventions in a similar way as the original complex system. We prove that for a class of nonlinear causal models, there exists a unique solution that achieves exact interventional consistency, ensuring learned explanations reflect meaningful causal patterns. Experiments on both synthetic causal models and practical RL tasks-including pendulum control and robot table tennis-demonstrate that our approach can uncover important behavioral patterns, biases, and failure modes in trained RL policies.