# The Procedural Content Generation Benchmark: An Open-source Testbed for Generative Challenges in Games

arXiv: https://arxiv.org/abs/2503.21474 · HTML: https://mlsanity.com/paper/2503.21474/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 56 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Builds on (most cited first, 25 of 25)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [Procedural Content Generation via Machine Learning (PCGML)](https://mlsanity.com/paper/1702.00539/) (2017)
- [Benchmarking and Survey of Explanation Methods for Black Box Models](https://mlsanity.com/paper/2102.13076/) (2021)
- [Large Language Models and Games: A Survey and Roadmap](https://mlsanity.com/paper/2402.18659/) (2024)
- [PCGRL: Procedural Content Generation via Reinforcement Learning](https://mlsanity.com/paper/2001.09212/) (2020)
- [Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Network](https://mlsanity.com/paper/1805.00728/) (2018)
- [General Video Game AI: a Multi-Track Framework for Evaluating Agents, Games and Content Generation Algorithms](https://mlsanity.com/paper/1802.10363/) (2018)
- [Bootstrapping Conditional GANs for Video Game Level Generation](https://mlsanity.com/paper/1910.01603/) (2019)
- [Talakat: Bullet Hell Generation through Constrained Map-Elites](https://mlsanity.com/paper/1806.04718/) (2018)
- [EvoCraft: A New Challenge for Open-Endedness](https://mlsanity.com/paper/2012.04751/) (2020)
- [Generative Design in Minecraft (GDMC), Settlement Generation Competition](https://mlsanity.com/paper/1803.09853/) (2018)
- [Baba is Y'all: Collaborative Mixed-Initiative Level Design](https://mlsanity.com/paper/2003.14294/) (2020)
- [Learning Controllable 3D Level Generators](https://mlsanity.com/paper/2206.13623/) (2022)
- [Path of Destruction: Learning an Iterative Level Generator Using a Small Dataset](https://mlsanity.com/paper/2202.10184/) (2022)
- [Transforming Exploratory Creativity with DeLeNoX](https://mlsanity.com/paper/2103.11715/) (2021)
- [Open-Ended Evolution for Minecraft Building Generation](https://mlsanity.com/paper/2209.03108/) (2022)
- [Procedural Content Generation using Neuroevolution and Novelty Search for Diverse Video Game Levels](https://mlsanity.com/paper/2204.06934/) (2022)
- [Toward Game Level Generation from Gameplay Videos](https://mlsanity.com/paper/1602.07721/) (2016)
- [10 Years of the PCG workshop: Past and Future Trends](https://mlsanity.com/paper/2104.11037/) (2021)
- [Preference-Learning Emitters for Mixed-Initiative Quality-Diversity Algorithms](https://mlsanity.com/paper/2210.13839/) (2022)
- [Creative Invention Benchmark](https://mlsanity.com/paper/1805.03720/) (2018)
- [Affectively Framework: Towards Human-like Affect-Based Agents](https://mlsanity.com/paper/2407.18316/) (2024)
- [ELIMINATION from Design to Analysis](https://mlsanity.com/paper/1905.06379/) (2019)

## Built on by (most cited first, 6 of 6)

- [90% Faster, 100% Code-Free: MLLM-Driven Zero-Code 3D Game Development](https://mlsanity.com/paper/2509.26161/) (2025)
- [Symbolically Scaffolded Play: Designing Role-Sensitive Prompts for Generative NPC Dialogue](https://mlsanity.com/paper/2510.25820/) (2025)
- [Evolutionary Wave Function Collapse](https://mlsanity.com/paper/2607.02082/) (2026)
- [Procedural Content Metageneration via Program Search and Continual Abstraction Discovery](https://mlsanity.com/paper/2608.17947/) (2026)
- [Programmable Cellular Automata](https://mlsanity.com/paper/2609.06102/) (2026)
- [Spec2Game: Can LLMs Generate Complete Playable Games from Detailed Specifications?](https://mlsanity.com/paper/2610.04253/) (2026)

## Abstract (arXiv)

This paper introduces the Procedural Content Generation Benchmark for evaluating generative algorithms on different game content creation tasks. The benchmark comes with 12 game-related problems with multiple variants on each problem. Problems vary from creating levels of different kinds to creating rule sets for simple arcade games. Each problem has its own content representation, control parameters, and evaluation metrics for quality, diversity, and controllability. This benchmark is intended as a first step towards a standardized way of comparing generative algorithms. We use the benchmark to score three baseline algorithms: a random generator, an evolution strategy, and a genetic algorithm. Results show that some problems are easier to solve than others, as well as the impact the chosen objective has on quality, diversity, and controllability of the generated artifacts.