# G-PCGRL: Procedural Graph Data Generation via Reinforcement Learning

arXiv: https://arxiv.org/abs/2407.10483 · HTML: https://mlsanity.com/paper/2407.10483/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 14 of 14)

- [Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation](https://mlsanity.com/paper/1806.02473/) (2018)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [Procedural Content Generation via Machine Learning (PCGML)](https://mlsanity.com/paper/1702.00539/) (2017)
- [PCGRL: Procedural Content Generation via Reinforcement Learning](https://mlsanity.com/paper/2001.09212/) (2020)
- [Deep Learning for Procedural Content Generation](https://mlsanity.com/paper/2010.04548/) (2020)
- [MarioGPT: Open-Ended Text2Level Generation through Large Language Models](https://mlsanity.com/paper/2302.05981/) (2023)
- [Learning Controllable Content Generators](https://mlsanity.com/paper/2105.02993/) (2021)
- [Adversarial Reinforcement Learning for Procedural Content Generation](https://mlsanity.com/paper/2103.04847/) (2021)
- [TOAD-GAN: Coherent Style Level Generation from a Single Example](https://mlsanity.com/paper/2008.01531/) (2020)
- [Learning Controllable 3D Level Generators](https://mlsanity.com/paper/2206.13623/) (2022)
- [Generative Adversarial Network Rooms in Generative Graph Grammar Dungeons for The Legend of Zelda](https://mlsanity.com/paper/2001.05065/) (2020)
- [TropeTwist: Trope-based Narrative Structure Generation](https://mlsanity.com/paper/2204.09672/) (2022)
- [Balancing of competitive two-player Game Levels with Reinforcement Learning](https://mlsanity.com/paper/2306.04429/) (2023)
- [GEEvo: Game Economy Generation and Balancing with Evolutionary Algorithms](https://mlsanity.com/paper/2404.18574/) (2024)

## Built on by (most cited first, 4 of 4)

- [IPCGRL: Language-Instructed Reinforcement Learning for Procedural Level Generation](https://mlsanity.com/paper/2503.12358/) (2025)
- [PCGRLLM: Large Language Model-Driven Reward Design for Procedural Content Generation Reinforcement Learning](https://mlsanity.com/paper/2502.10906/) (2025)
- [Human-Aligned Procedural Level Generation Reinforcement Learning via Text-Level-Sketch Shared Representation](https://mlsanity.com/paper/2508.09860/) (2025)
- [Level the Level: Balancing Game Levels for Asymmetric Player Archetypes With Reinforcement Learning](https://mlsanity.com/paper/2503.24099/) (2025)

## Abstract (arXiv)

Graph data structures offer a versatile and powerful means to model relationships and interconnections in various domains, promising substantial advantages in data representation, analysis, and visualization. In games, graph-based data structures are omnipresent and represent, for example, game economies, skill trees or complex, branching quest lines. With this paper, we propose G-PCGRL, a novel and controllable method for the procedural generation of graph data using reinforcement learning. Therefore, we frame this problem as manipulating a graph's adjacency matrix to fulfill a given set of constraints. Our method adapts and extends the Procedural Content Generation via Reinforcement Learning (PCGRL) framework and introduces new representations to frame the problem of graph data generation as a Markov decision process. We compare the performance of our method with the original PCGRL, the run time with a random search and evolutionary algorithm, and evaluate G-PCGRL on two graph data domains in games: game economies and skill trees. The results show that our method is capable of generating graph-based content quickly and reliably to support and inspire designers in the game creation process. In addition, trained models are controllable in terms of the type and number of nodes to be generated.