# Video Game Level Design as a Multi-Agent Reinforcement Learning Problem

arXiv: https://arxiv.org/abs/2510.04862 · HTML: https://mlsanity.com/paper/2510.04862/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 6 Oct 2025.

## Builds on (most cited first, 24 of 24)

- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design](https://mlsanity.com/paper/2012.02096/) (2020)
- [Evolving Curricula with Regret-Based Environment Design](https://mlsanity.com/paper/2203.01302/) (2022)
- [Procedural Content Generation via Machine Learning (PCGML)](https://mlsanity.com/paper/1702.00539/) (2017)
- [PCGRL: Procedural Content Generation via Reinforcement Learning](https://mlsanity.com/paper/2001.09212/) (2020)
- [JaxMARL: Multi-Agent RL Environments and Algorithms in JAX](https://mlsanity.com/paper/2311.10090/) (2023)
- [Guided Deep Reinforcement Learning for Swarm Systems](https://mlsanity.com/paper/1709.06011/) (2017)
- [Super Mario as a String: Platformer Level Generation Via LSTMs](https://mlsanity.com/paper/1603.00930/) (2016)
- [Learning Controllable Content Generators](https://mlsanity.com/paper/2105.02993/) (2021)
- [Bootstrapping Conditional GANs for Video Game Level Generation](https://mlsanity.com/paper/1910.01603/) (2019)
- [Friend, Collaborator, Student, Manager: How Design of an AI-Driven Game Level Editor Affects Creators](https://mlsanity.com/paper/1901.06417/) (2019)
- [Generating and Blending Game Levels via Quality-Diversity in the Latent Space of a Variational Autoencoder](https://mlsanity.com/paper/2102.12463/) (2021)
- [Interactive Evolution and Exploration Within Latent Level-Design Space of Generative Adversarial Networks](https://mlsanity.com/paper/2004.00151/) (2020)
- [Baba is Y'all: Collaborative Mixed-Initiative Level Design](https://mlsanity.com/paper/2003.14294/) (2020)
- [Co-Creative Level Design via Machine Learning](https://mlsanity.com/paper/1809.09420/) (2018)
- [The Five-Dollar Model: Generating Game Maps and Sprites from Sentence Embeddings](https://mlsanity.com/paper/2308.04052/) (2023)
- [Mixed-Initiative Level Design with RL Brush](https://mlsanity.com/paper/2008.02778/) (2020)
- [JaxUED: A simple and useable UED library in Jax](https://mlsanity.com/paper/2403.13091/) (2024)
- [DreamGarden: A Designer Assistant for Growing Games from a Single Prompt](https://mlsanity.com/paper/2410.01791/) (2024)
- [PCGRL+: Scaling, Control and Generalization in Reinforcement Learning Level Generators](https://mlsanity.com/paper/2408.12525/) (2024)
- [Making New Connections: LLMs as Puzzle Generators for The New York Times' Connections Word Game](https://mlsanity.com/paper/2407.11240/) (2024)
- [Machine learning based co-creative design framework](https://mlsanity.com/paper/2001.08791/) (2020)
- [Procedural city modeling](https://mlsanity.com/paper/2507.18899/) (2025)

## Built on by (most cited first, 1 of 1)

- [RL-Lock: Reinforcement Learning for Generating Interlocking Assemblies](https://mlsanity.com/paper/2608.01744/) (2026)

## Abstract (arXiv)

Procedural Content Generation via Reinforcement Learning (PCGRL) offers a method for training controllable level designer agents without the need for human datasets, using metrics that serve as proxies for level quality as rewards. Existing PCGRL research focuses on single generator agents, but are bottlenecked by the need to frequently recalculate heuristics of level quality and the agent's need to navigate around potentially large maps. By framing level generation as a multi-agent problem, we mitigate the efficiency bottleneck of single-agent PCGRL by reducing the number of reward calculations relative to the number of agent actions. We also find that multi-agent level generators are better able to generalize to out-of-distribution map shapes, which we argue is due to the generators' learning more local, modular design policies. We conclude that treating content generation as a distributed, multi-agent task is beneficial for generating functional artifacts at scale.