# Graph Contextual Reinforcement Learning for Efficient Directed Controller Synthesis

arXiv: https://arxiv.org/abs/2512.15295 · HTML: https://mlsanity.com/paper/2512.15295/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 2 of its 24 references that are in the map.
- First posted on arXiv 17 Dec 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 2 of 2)

- [Reinforcement Learning for Classical Planning: Viewing Heuristics as Dense Reward Generators](https://mlsanity.com/paper/2109.14830/) (2021)
- [Exploration Policies for On-the-Fly Controller Synthesis: A Reinforcement Learning Approach](https://mlsanity.com/paper/2210.05393/) (2022)

## Abstract (arXiv)

Controller synthesis is a formal method approach for automatically generating Labeled Transition System (LTS) controllers that satisfy specified properties. The efficiency of the synthesis process, however, is critically dependent on exploration policies. These policies often rely on fixed rules or strategies learned through reinforcement learning (RL) that consider only a limited set of current features. To address this limitation, this paper introduces GCRL, an approach that enhances RL-based methods by integrating Graph Neural Networks (GNNs). GCRL encodes the history of LTS exploration into a graph structure, allowing it to capture a broader, non-current-based context. In a comparative experiment against state-of-the-art methods, GCRL exhibited superior learning efficiency and generalization across four out of five benchmark domains, except one particular domain characterized by high symmetry and strictly local interactions.