# Property-driven Causal Abstractions for Markov Decision Processes

arXiv: https://arxiv.org/abs/2607.26787 · HTML: https://mlsanity.com/paper/2607.26787/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 51 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Builds on (most cited first, 22 of 22)

- [Invariant Causal Prediction for Block MDPs](https://mlsanity.com/paper/2003.06016/) (2020)
- [The Probabilistic Model Checker Storm](https://mlsanity.com/paper/2002.07080/) (2020)
- [Causal Dynamics Learning for Task-Independent State Abstraction](https://mlsanity.com/paper/2206.13452/) (2022)
- [A Modification of the Halpern-Pearl Definition of Causality](https://mlsanity.com/paper/1505.00162/) (2015)
- [A Survey on Causal Reinforcement Learning](https://mlsanity.com/paper/2302.05209/) (2023)
- [Causal Reinforcement Learning using Observational and Interventional Data](https://mlsanity.com/paper/2106.14421/) (2021)
- [Causal Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.01452/) (2023)
- [Robust Markov Decision Processes: A Place Where AI and Formal Methods Meet](https://mlsanity.com/paper/2411.11451/) (2024)
- [Causality in Configurable Software Systems](https://mlsanity.com/paper/2201.07280/) (2022)
- [From Verification to Causality-based Explications](https://mlsanity.com/paper/2105.09533/) (2021)
- [Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning](https://mlsanity.com/paper/2401.12497/) (2024)
- [Decision-Making Under Uncertainty: Beyond Probabilities](https://mlsanity.com/paper/2303.05848/) (2023)
- [Strategy Synthesis for Partially-known Switched Stochastic Systems](https://mlsanity.com/paper/2104.02172/) (2021)
- [Toward Causal-Aware RL: State-Wise Action-Refined Temporal Difference](https://mlsanity.com/paper/2201.00354/) (2022)
- [Probabilistic causes in Markov chains](https://mlsanity.com/paper/2104.13604/) (2021)
- [HyperPCTL: A Temporal Logic for Probabilistic Hyperproperties](https://mlsanity.com/paper/1804.01853/) (2018)
- [Causal Temporal Reasoning for Markov Decision Processes](https://mlsanity.com/paper/2212.08712/) (2022)
- [Robust Counterfactual Inference in Markov Decision Processes](https://mlsanity.com/paper/2502.13731/) (2025)
- [Foundations of probability-raising causality in Markov decision processes](https://mlsanity.com/paper/2209.02973/) (2022)
- [Test Where Decisions Matter: Importance-driven Testing for Deep Reinforcement Learning](https://mlsanity.com/paper/2411.07700/) (2024)
- [Probability-Raising Causality for Uncertain Parametric Markov Decision Processes with PAC Guarantees](https://mlsanity.com/paper/2507.07319/) (2025)
- [PrIC3: Property Directed Reachability for MDPs](https://mlsanity.com/paper/2004.14835/) (2020)

## Abstract (arXiv)

Markov Decision Processes (MDPs) are widely used as decision-making models, commonly specified over factored state spaces through state variables and their valuations. The exponential blowup in the number of states renders many reasoning tasks in MDPs challenging. Abstractions are promising techniques to reduce MDPs and thus mitigate scalability issues. In this work, we introduce a notion of causality on factored MDPs and a novel property-driven causal abstraction technique that retains many characteristics of the original MDP model. For this, we rely on causal relations over state variable predicates and identify those states that share the same reasons for fulfilling or violating a given abstraction property. We theoretically and empirically compare various causal MDP abstractions using different model types such as MDPs, interval MDPs, or stochastic games. Our evaluation demonstrates the potential of our approach: For several standard benchmarks, we obtain small abstractions that allow us to compute near-optimal policies for the original MDP. Furthermore, our causal abstractions often generalize to related large-scale MDP models.