# Automated Play-Testing Through RL Based Human-Like Play-Styles Generation

arXiv: https://arxiv.org/abs/2211.17188 · HTML: https://mlsanity.com/paper/2211.17188/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 29 Nov 2022.

## Builds on (most cited first, 13 of 13)

- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [Sample Efficient Actor-Critic with Experience Replay](https://mlsanity.com/paper/1611.01224/) (2016)
- [Automatic Curriculum Learning For Deep RL: A Short Survey](https://mlsanity.com/paper/2003.04664/) (2020)
- [Goal-Conditioned Reinforcement Learning with Imagined Subgoals](https://mlsanity.com/paper/2107.00541/) (2021)
- [Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments](https://mlsanity.com/paper/1910.07224/) (2019)
- [Modeling Strong and Human-Like Gameplay with KL-Regularized Search](https://mlsanity.com/paper/2112.07544/) (2021)
- [Augmenting Automated Game Testing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.15819/) (2021)
- [Direct Behavior Specification via Constrained Reinforcement Learning](https://mlsanity.com/paper/2112.12228/) (2021)
- [Automated Playtesting with Procedural Personas through MCTS with Evolved Heuristics](https://mlsanity.com/paper/1802.06881/) (2018)
- [Deep Reinforcement Learning for Navigation in AAA Video Games](https://mlsanity.com/paper/2011.04764/) (2020)
- [CCPT: Automatic Gameplay Testing and Validation with Curiosity-Conditioned Proximal Trajectories](https://mlsanity.com/paper/2202.10057/) (2022)
- [Inverse Reinforcement Learning from Summary Data](https://mlsanity.com/paper/1703.09700/) (2017)

## Built on by (most cited first, 11 of 11)

- [Mechanic Maker 2.0: Reinforcement Learning for Evaluating Generated Rules](https://mlsanity.com/paper/2309.09476/) (2023)
- [Towards Human-Like RL: Taming Non-Naturalistic Behavior in Deep RL via Adaptive Behavioral Costs in 3D Games](https://mlsanity.com/paper/2309.15484/) (2023)
- [Adversarial Imitation Learning On Aggregated Data](https://mlsanity.com/paper/2311.08568/) (2023)
- [Cultivating Game Sense for Yourself: Making VLMs Gaming Experts](https://mlsanity.com/paper/2503.21263/) (2025)
- [A modular framework for automated evaluation of procedural content generation in serious games with deep reinforcement learning agents](https://mlsanity.com/paper/2505.16801/) (2025)
- [UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors](https://mlsanity.com/paper/2512.10835/) (2025)
- [Robo-Saber: Generating and Simulating Virtual Reality Players](https://mlsanity.com/paper/2602.18319/) (2026)
- [CA2: Code-Aware Agent for Automated Game Testing](https://mlsanity.com/paper/2605.13918/) (2026)
- [Coachable agents for interactive gameplay](https://mlsanity.com/paper/2607.00642/) (2026)
- [Reward-Adaptive Iterative Discovery: A Case Study on Automated Game Testing for NHL26](https://mlsanity.com/paper/2607.07498/) (2026)
- [Comparative Analysis of GAT and BERT for Human-Like Playtesting](https://mlsanity.com/paper/2607.11501/) (2026)

## Abstract (arXiv)

The increasing complexity of gameplay mechanisms in modern video games is leading to the emergence of a wider range of ways to play games. The variety of possible play-styles needs to be anticipated by designers, through automated tests. Reinforcement Learning is a promising answer to the need of automating video game testing. To that effect one needs to train an agent to play the game, while ensuring this agent will generate the same play-styles as the players in order to give meaningful feedback to the designers. We present CARMI: a Configurable Agent with Relative Metrics as Input. An agent able to emulate the players play-styles, even on previously unseen levels. Unlike current methods it does not rely on having full trajectories, but only summary data. Moreover it only requires little human data, thus compatible with the constraints of modern video game production. This novel agent could be used to investigate behaviors and balancing during the production of a video game with a realistic amount of training time.