# Increasing Generality in Machine Learning through Procedural Content Generation

arXiv: https://arxiv.org/abs/1911.13071 · HTML: https://mlsanity.com/paper/1911.13071/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2019.

## Built on by (most cited first, 30 of 59)

- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [Evolving Curricula with Regret-Based Environment Design](https://mlsanity.com/paper/2203.01302/) (2022)
- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Human-Timescale Adaptation in an Open-Ended Task Space](https://mlsanity.com/paper/2301.07608/) (2023)
- [REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2505.24760/) (2025)
- [SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2212.07489/) (2022)
- [Replay-Guided Adversarial Environment Design](https://mlsanity.com/paper/2110.02439/) (2021)
- [Video as the New Language for Real-World Decision Making](https://mlsanity.com/paper/2402.17139/) (2024)
- [MarioGPT: Open-Ended Text2Level Generation through Large Language Models](https://mlsanity.com/paper/2302.05981/) (2023)
- [Deep Surrogate Assisted Generation of Environments](https://mlsanity.com/paper/2206.04199/) (2022)
- [Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agents](https://mlsanity.com/paper/2102.02926/) (2021)
- [Adversarial Reinforcement Learning for Procedural Content Generation](https://mlsanity.com/paper/2103.04847/) (2021)
- [Experience-Driven PCG via Reinforcement Learning: A Super Mario Bros Study](https://mlsanity.com/paper/2106.15877/) (2021)
- [Procedural Content Generation in Games: A Survey with Insights on Emerging LLM Integration](https://mlsanity.com/paper/2410.15644/) (2024)
- [General Intelligence Requires Rethinking Exploration](https://mlsanity.com/paper/2211.07819/) (2022)
- [Emergent Bartering Behaviour in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.06760/) (2022)
- [Multi-Robot Coordination and Layout Design for Automated Warehousing](https://mlsanity.com/paper/2305.06436/) (2023)
- [DreamCraft: Text-Guided Generation of Functional 3D Environments in Minecraft](https://mlsanity.com/paper/2404.15538/) (2024)
- [Games for Artificial Intelligence Research: A Review and Perspectives](https://mlsanity.com/paper/2304.13269/) (2023)
- [Inferring Capabilities from Task Performance with Bayesian Triangulation](https://mlsanity.com/paper/2309.11975/) (2023)
- [Learning General World Models in a Handful of Reward-Free Deployments](https://mlsanity.com/paper/2210.12719/) (2022)
- [Arbitrarily Scalable Environment Generators via Neural Cellular Automata](https://mlsanity.com/paper/2310.18622/) (2023)
- [Surrogate Assisted Generation of Human-Robot Interaction Scenarios](https://mlsanity.com/paper/2304.13787/) (2023)
- [Demonstration-efficient Inverse Reinforcement Learning in Procedurally Generated Environments](https://mlsanity.com/paper/2012.02527/) (2020)
- [L2Explorer: A Lifelong Reinforcement Learning Assessment Environment](https://mlsanity.com/paper/2203.07454/) (2022)
- [Variational Offline Multi-agent Skill Discovery](https://mlsanity.com/paper/2405.16386/) (2024)
- [Causally Aligned Curriculum Learning](https://mlsanity.com/paper/2503.16799/) (2025)
- [BehAVE: Behaviour Alignment of Video Game Encodings](https://mlsanity.com/paper/2402.01335/) (2024)
- [Measuring Diversity of Game Scenarios](https://mlsanity.com/paper/2404.15192/) (2024)
- [GriddlyJS: A Web IDE for Reinforcement Learning](https://mlsanity.com/paper/2207.06105/) (2022)

## Abstract (arXiv)

Procedural Content Generation (PCG) refers to the practice, in videogames and other games, of generating content such as levels, quests, or characters algorithmically. Motivated by the need to make games replayable, as well as to reduce authoring burden, limit storage space requirements, and enable particular aesthetics, a large number of PCG methods have been devised by game developers. Additionally, researchers have explored adapting methods from machine learning, optimization, and constraint solving to PCG problems. Games have been widely used in AI research since the inception of the field, and in recent years have been used to develop and benchmark new machine learning algorithms. Through this practice, it has become more apparent that these algorithms are susceptible to overfitting. Often, an algorithm will not learn a general policy, but instead a policy that will only work for a particular version of a particular task with particular initial parameters. In response, researchers have begun exploring randomization of problem parameters to counteract such overfitting and to allow trained policies to more easily transfer from one environment to another, such as from a simulated robot to a robot in the real world. Here we review the large amount of existing work on PCG, which we believe has an important role to play in increasing the generality of machine learning methods. The main goal here is to present RL/AI with new tools from the PCG toolbox, and its secondary goal is to explain to game developers and researchers a way in which their work is relevant to AI research.