# Learning Local Constraints for Reinforcement-Learned Content Generators

arXiv: https://arxiv.org/abs/2605.13570 · HTML: https://mlsanity.com/paper/2605.13570/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 13 May 2026.

## Builds on (most cited first, 21 of 21)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [A Closer Look at Invalid Action Masking in Policy Gradient Algorithms](https://mlsanity.com/paper/2006.14171/) (2020)
- [Procedural Content Generation via Machine Learning (PCGML)](https://mlsanity.com/paper/1702.00539/) (2017)
- [PCGRL: Procedural Content Generation via Reinforcement Learning](https://mlsanity.com/paper/2001.09212/) (2020)
- [Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Network](https://mlsanity.com/paper/1805.00728/) (2018)
- [Level Generation Through Large Language Models](https://mlsanity.com/paper/2302.05817/) (2023)
- [The VGLC: The Video Game Level Corpus](https://mlsanity.com/paper/1606.07487/) (2016)
- [Super Mario as a String: Platformer Level Generation Via LSTMs](https://mlsanity.com/paper/1603.00930/) (2016)
- [Learning Controllable Content Generators](https://mlsanity.com/paper/2105.02993/) (2021)
- [Growing 3D Artefacts and Functional Machines with Neural Cellular Automata](https://mlsanity.com/paper/2103.08737/) (2021)
- [Adversarial Reinforcement Learning for Procedural Content Generation](https://mlsanity.com/paper/2103.04847/) (2021)
- [Experience-Driven PCG via Reinforcement Learning: A Super Mario Bros Study](https://mlsanity.com/paper/2106.15877/) (2021)
- [Interactive Evolution and Exploration Within Latent Level-Design Space of Generative Adversarial Networks](https://mlsanity.com/paper/2004.00151/) (2020)
- [Learning Controllable 3D Level Generators](https://mlsanity.com/paper/2206.13623/) (2022)
- [Tile Pattern KL-Divergence for Analysing and Evolving Game Levels](https://mlsanity.com/paper/1905.05077/) (2019)
- [Controllable Level Blending between Games using Variational Autoencoders](https://mlsanity.com/paper/2002.11869/) (2020)
- [Illuminating the Space of Beatable Lode Runner Levels Produced By Various Generative Adversarial Networks](https://mlsanity.com/paper/2101.07868/) (2021)
- [Path of Destruction: Learning an Iterative Level Generator Using a Small Dataset](https://mlsanity.com/paper/2202.10184/) (2022)
- [Multi-Domain Level Generation and Blending with Sketches via Example-Driven BSP and Variational Autoencoders](https://mlsanity.com/paper/2006.09807/) (2020)
- [Addressing the Fundamental Tension of PCGML with Discriminative Learning](https://mlsanity.com/paper/1809.04432/) (2018)
- [Generating Lode Runner Levels by Learning Player Paths with LSTMs](https://mlsanity.com/paper/2107.12532/) (2021)

## Abstract (arXiv)

Constraint-based game content generators that learn local constraints from existing content, such as Wave Function Collapse (WFC), can generate visually satisfying game levels but face challenges in optimizing global properties, such as playability. On the other hand, reinforcement-learning-trained generators can optimize global properties---because such properties can easily be included in reward functions---but the results can be visually dissatisfying. In this paper, we explore ways to combine these methods. Specifically, we constrain the action space of a PCGRL generator with constraints learned by WFC, effectively allowing the PCGRL generator to achieve global properties while being forced to adhere to local constraints. To better analyze how this hybrid content generation method operates, we vary the number and type of inputs, and we test whether to randomly collapse the starting state and exclude rare patterns. While the method is sensitive to hyperparameter tuning, the best of our trained generators produce visually satisfying and playable puzzle-platform game levels---such as Lode Runner levels---with desired global properties.