# Balancing of competitive two-player Game Levels with Reinforcement Learning

arXiv: https://arxiv.org/abs/2306.04429 · HTML: https://mlsanity.com/paper/2306.04429/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Procedural Content Generation via Machine Learning (PCGML)](https://mlsanity.com/paper/1702.00539/) (2017)
- [PCGRL: Procedural Content Generation via Reinforcement Learning](https://mlsanity.com/paper/2001.09212/) (2020)
- [Deep Learning for Procedural Content Generation](https://mlsanity.com/paper/2010.04548/) (2020)
- [Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents](https://mlsanity.com/paper/1903.00784/) (2019)
- [Learning Controllable Content Generators](https://mlsanity.com/paper/2105.02993/) (2021)
- [Adversarial Reinforcement Learning for Procedural Content Generation](https://mlsanity.com/paper/2103.04847/) (2021)
- [TOAD-GAN: Coherent Style Level Generation from a Single Example](https://mlsanity.com/paper/2008.01531/) (2020)
- [Learning Controllable 3D Level Generators](https://mlsanity.com/paper/2206.13623/) (2022)
- [Learning to Generate Levels From Nothing](https://mlsanity.com/paper/2002.05259/) (2020)
- [DOOM Level Generation using Generative Adversarial Networks](https://mlsanity.com/paper/1804.09154/) (2018)
- [World-GAN: a Generative Model for Minecraft Worlds](https://mlsanity.com/paper/2106.10155/) (2021)
- [Path of Destruction: Learning an Iterative Level Generator Using a Small Dataset](https://mlsanity.com/paper/2202.10184/) (2022)
- [Mutation Models: Learning to Generate Levels by Imitating Evolution](https://mlsanity.com/paper/2206.05497/) (2022)
- [Generating Real-Time Strategy Game Units Using Search-Based Procedural Content Generation and Monte Carlo Tree Search](https://mlsanity.com/paper/2212.03387/) (2022)

## Built on by (most cited first, 4 of 4)

- [G-PCGRL: Procedural Graph Data Generation via Reinforcement Learning](https://mlsanity.com/paper/2407.10483/) (2024)
- [Identifying and Clustering Counter Relationships of Team Compositions in PvP Games for Efficient Balance Analysis](https://mlsanity.com/paper/2408.17180/) (2024)
- [Simulation-Driven Balancing of Competitive Game Levels with Reinforcement Learning](https://mlsanity.com/paper/2503.18748/) (2025)
- [Level the Level: Balancing Game Levels for Asymmetric Player Archetypes With Reinforcement Learning](https://mlsanity.com/paper/2503.24099/) (2025)

## Abstract (arXiv)

The balancing process for game levels in a competitive two-player context involves a lot of manual work and testing, particularly in non-symmetrical game levels. In this paper, we propose an architecture for automated balancing of tile-based levels within the recently introduced PCGRL framework (procedural content generation via reinforcement learning). Our architecture is divided into three parts: (1) a level generator, (2) a balancing agent and, (3) a reward modeling simulation. By playing the level in a simulation repeatedly, the balancing agent is rewarded for modifying it towards the same win rates for all players. To this end, we introduce a novel family of swap-based representations to increase robustness towards playability. We show that this approach is capable to teach an agent how to alter a level for balancing better and faster than plain PCGRL. In addition, by analyzing the agent's swapping behavior, we can draw conclusions about which tile types influence the balancing most. We test and show our results using the Neural MMO (NMMO) environment in a competitive two-player setting.