# CRRL: A Causality-Based Reinforcement Learning Framework for Autonomous System Recovery

arXiv: https://arxiv.org/abs/2607.03177 · HTML: https://mlsanity.com/paper/2607.03177/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 47 references that are in the map.
- First posted on arXiv 3 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 18 of 18)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Detecting causal associations in large nonlinear time series datasets](https://mlsanity.com/paper/1702.07007/) (2017)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?](https://mlsanity.com/paper/2006.14911/) (2020)
- [Causal Reinforcement Learning using Observational and Interventional Data](https://mlsanity.com/paper/2106.14421/) (2021)
- [Deep Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1811.11329/) (2018)
- [Causal Multi-Agent Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2111.06721/) (2021)
- [Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Traffic](https://mlsanity.com/paper/2202.00881/) (2022)
- [Causal Imitative Model for Autonomous Driving](https://mlsanity.com/paper/2112.03908/) (2021)
- [Uncertainty in Self-Adaptive Systems: A Research Community Perspective](https://mlsanity.com/paper/2103.02717/) (2021)
- [Evaluating Temporal Observation-Based Causal Discovery Techniques Applied to Road Driver Behaviour](https://mlsanity.com/paper/2302.00064/) (2023)
- [Simulation-Based Counterfactual Causal Discovery on Real World Driver Behaviour](https://mlsanity.com/paper/2306.03354/) (2023)
- [Improving Scalability and Reward of Utility-Driven Self-Healing for Large Dynamic Architectures](https://mlsanity.com/paper/2005.11180/) (2020)
- [Deep Learning for Effective and Efficient Reduction of Large Adaptation Spaces in Self-Adaptive Systems](https://mlsanity.com/paper/2204.06254/) (2022)
- [TraffNet: Learning Causality of Traffic Generation for What-if Prediction](https://mlsanity.com/paper/2303.15954/) (2023)
- [Extending Structural Causal Models for Autonomous Vehicles to Simplify Temporal System Construction & Enable Dynamic Interactions Between Agents](https://mlsanity.com/paper/2406.01384/) (2024)

## Abstract (arXiv)

Traditional reinforcement learning (RL) for recovery in autonomous systems lacks causal understanding and generalizes poorly to novel failure scenarios. RL policies often stall in failure states, spending up to 70% of an episode immobilized. Rule-based recovery alone is inadequate, and adding heuristic recovery to a pretrained PPO policy worsens rewards because policies cannot coordinate well with unanticipated interventions. The issue is not missing recovery mechanisms but a lack of policies trained to collaborate with them. We introduce CRRL, a causal-guided RL framework that trains policies to work effectively with rule-based recovery. The recovery detects stalled states and assists the agent. Causal relations from driving logs shape the training signal, teaching the policy to anticipate stalls and adjust actions in recovery contexts. The framework follows MAPE-K, with sensor collection, causal model construction, and hybrid RL policy training corresponding to Monitor, Analyze, and Plan/Execute, respectively. We evaluate CRRL through a four-condition ablation study across three driving scenarios, with 20 episodes per condition. We find that causal training significantly improves reward, distance, and velocity. Moreover, 9 of 20 roundabout episodes required zero recovery intervention, confirming navigation competence. These results show that causal-guided training produces effective RL policies that cooperate with rule-based safety components.