# Inferring Causal Graph Temporal Logic Formulas to Expedite Reinforcement Learning in Temporally Extended Tasks

arXiv: https://arxiv.org/abs/2601.02666 · HTML: https://mlsanity.com/paper/2601.02666/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 2 of its 6 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 2 of 2)

- [Policy Synthesis for Factored MDPs with Graph Temporal Logic Specifications](https://mlsanity.com/paper/2001.09066/) (2020)
- [Expediting Reinforcement Learning by Incorporating Knowledge About Temporal Causality in the Environment](https://mlsanity.com/paper/2510.15456/) (2025)

## Abstract (arXiv)

Decision-making tasks often unfold on graphs with spatial-temporal dynamics. Black-box reinforcement learning often overlooks how local changes spread through network structure, limiting sample efficiency and interpretability. We present GTL-CIRL, a closed-loop framework that simultaneously learns policies and mines Causal Graph Temporal Logic (Causal GTL) specifications. The method shapes rewards with robustness, collects counterexamples when effects fail, and uses Gaussian Process (GP) driven Bayesian optimization to refine parameterized cause templates. The GP models capture spatial and temporal correlations in the system dynamics, enabling efficient exploration of complex parameter spaces. Case studies in gene and power networks show faster learning and clearer, verifiable behavior compared to standard RL baselines.