# Simulation-Driven Balancing of Competitive Game Levels with Reinforcement Learning

arXiv: https://arxiv.org/abs/2503.18748 · HTML: https://mlsanity.com/paper/2503.18748/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 23 of its 50 references that are in the map.
- First posted on arXiv 24 Mar 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 23 of 23)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Aequitas: A Bias and Fairness Audit Toolkit](https://mlsanity.com/paper/1811.05577/) (2018)
- [Procedural Content Generation via Machine Learning (PCGML)](https://mlsanity.com/paper/1702.00539/) (2017)
- [PCGRL: Procedural Content Generation via Reinforcement Learning](https://mlsanity.com/paper/2001.09212/) (2020)
- [Deep Learning for Procedural Content Generation](https://mlsanity.com/paper/2010.04548/) (2020)
- [Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents](https://mlsanity.com/paper/1903.00784/) (2019)
- [Learning Controllable Content Generators](https://mlsanity.com/paper/2105.02993/) (2021)
- [Adversarial Reinforcement Learning for Procedural Content Generation](https://mlsanity.com/paper/2103.04847/) (2021)
- [TOAD-GAN: Coherent Style Level Generation from a Single Example](https://mlsanity.com/paper/2008.01531/) (2020)
- [Learning Controllable 3D Level Generators](https://mlsanity.com/paper/2206.13623/) (2022)
- [Learning to Generate Levels From Nothing](https://mlsanity.com/paper/2002.05259/) (2020)
- [DOOM Level Generation using Generative Adversarial Networks](https://mlsanity.com/paper/1804.09154/) (2018)
- [Path of Destruction: Learning an Iterative Level Generator Using a Small Dataset](https://mlsanity.com/paper/2202.10184/) (2022)
- [Mutation Models: Learning to Generate Levels by Imitating Evolution](https://mlsanity.com/paper/2206.05497/) (2022)
- [Evolving the Hearthstone Meta](https://mlsanity.com/paper/1907.01623/) (2019)
- [TropeTwist: Trope-based Narrative Structure Generation](https://mlsanity.com/paper/2204.09672/) (2022)
- [Demonstrating the Feasibility of Automatic Game Balancing](https://mlsanity.com/paper/1603.03795/) (2016)
- [Generating Real-Time Strategy Game Units Using Search-Based Procedural Content Generation and Monte Carlo Tree Search](https://mlsanity.com/paper/2212.03387/) (2022)
- [Learning of Generalizable and Interpretable Knowledge in Grid-Based Reinforcement Learning Environments](https://mlsanity.com/paper/2309.03651/) (2023)
- [Balancing of competitive two-player Game Levels with Reinforcement Learning](https://mlsanity.com/paper/2306.04429/) (2023)
- [GEEvo: Game Economy Generation and Balancing with Evolutionary Algorithms](https://mlsanity.com/paper/2404.18574/) (2024)

## Abstract (arXiv)

The balancing process for game levels in competitive two-player contexts involves a lot of manual work and testing, particularly for non-symmetrical game levels. In this work, we frame game balancing as a procedural content generation task and propose an architecture for automatically balancing of tile-based levels within the PCGRL framework (procedural content generation via reinforcement learning). Our architecture is divided into three parts: (1) a level generator, (2) a balancing agent, and (3) a reward modeling simulation. Through repeated simulations, the balancing agent receives rewards for adjusting the level towards a given balancing objective, such as equal win rates for all players. To this end, we propose new swap-based representations to improve the robustness of playability, thereby enabling agents to balance game levels more effectively and quickly compared to traditional PCGRL. By analyzing the agent's swapping behavior, we can infer which tile types have the most impact on the balance. We validate our approach in the Neural MMO (NMMO) environment in a competitive two-player scenario. In this extended conference paper, we present improved results, explore the applicability of the method to various forms of balancing beyond equal balancing, compare the performance to another search-based approach, and discuss the application of existing fairness metrics to game balancing.