# Improving Playtesting Coverage via Curiosity Driven Reinforcement Learning Agents

arXiv: https://arxiv.org/abs/2103.13798 · HTML: https://mlsanity.com/paper/2103.13798/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 25 Mar 2021.

## Builds on (most cited first, 6 of 6)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Unifying Count-Based Exploration and Intrinsic Motivation](https://mlsanity.com/paper/1606.01868/) (2016)
- [A survey on intrinsic motivation in reinforcement learning](https://mlsanity.com/paper/1908.06976/) (2019)
- [Automated Playtesting with Procedural Personas through MCTS with Evolved Heuristics](https://mlsanity.com/paper/1802.06881/) (2018)
- [Taking the Scenic Route: Automatic Exploration for Videogames](https://mlsanity.com/paper/1812.03125/) (2018)

## Built on by (most cited first, 28 of 28)

- [Direct Behavior Specification via Constrained Reinforcement Learning](https://mlsanity.com/paper/2112.12228/) (2021)
- [CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learning](https://mlsanity.com/paper/2203.11096/) (2022)
- [Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learning](https://mlsanity.com/paper/2208.07811/) (2022)
- [Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Games](https://mlsanity.com/paper/2307.11105/) (2023)
- [Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation](https://mlsanity.com/paper/2207.08379/) (2022)
- [CCPT: Automatic Gameplay Testing and Validation with Curiosity-Conditioned Proximal Trajectories](https://mlsanity.com/paper/2202.10057/) (2022)
- [Learning to Identify Perceptual Bugs in 3D Video Games](https://mlsanity.com/paper/2202.12884/) (2022)
- [Giving Feedback on Interactive Student Programs with Meta-Exploration](https://mlsanity.com/paper/2211.08802/) (2022)
- [Graph augmented Deep Reinforcement Learning in the GameRLand3D environment](https://mlsanity.com/paper/2112.11731/) (2021)
- [Godot Reinforcement Learning Agents](https://mlsanity.com/paper/2112.03636/) (2021)
- [Improving Generalization in Game Agents with Data Augmentation in Imitation Learning](https://mlsanity.com/paper/2309.12815/) (2023)
- [Play to Grade: Testing Coding Games as Classifying Markov Decision Process](https://mlsanity.com/paper/2110.14615/) (2021)
- [World of Bugs: A Platform for Automated Bug Detection in 3D Video Games](https://mlsanity.com/paper/2206.11037/) (2022)
- [Go-Explore Complex 3D Game Environments for Automated Reachability Testing](https://mlsanity.com/paper/2209.00570/) (2022)
- [Generating Personas for Games with Multimodal Adversarial Imitation Learning](https://mlsanity.com/paper/2308.07598/) (2023)
- [Preference-conditioned Pixel-based AI Agent For Game Testing](https://mlsanity.com/paper/2308.09289/) (2023)
- [Bayesian Optimization-based Search for Agent Control in Automated Game Testing](https://mlsanity.com/paper/2508.13121/) (2025)
- [AI for Games in the Foundation Model Era](https://mlsanity.com/paper/2609.16679/) (2026)
- [Playtesting: What is Beyond Personas](https://mlsanity.com/paper/2107.11965/) (2021)
- [Automatic Testing and Validation of Level of Detail Reductions Through Supervised Learning](https://mlsanity.com/paper/2208.12674/) (2022)
- [Synergizing Code Coverage and Gameplay Intent: Coverage-Aware Game Playtesting with LLM-Guided Reinforcement Learning](https://mlsanity.com/paper/2512.12706/) (2025)
- [Efficient Exploration via First-Person Behavior Cloning Assisted Rapidly-Exploring Random Trees](https://mlsanity.com/paper/2203.12774/) (2022)
- [Automatic Bug Detection in Games using LSTM Networks](https://mlsanity.com/paper/2312.08418/) (2023)
- [Knowledge Graph-enhanced Large Language Model for Incremental Game PlayTesting](https://mlsanity.com/paper/2511.02534/) (2025)
- [Cooperative Multi-agent Approach for Automated Computer Game Testing](https://mlsanity.com/paper/2405.11347/) (2024)
- [UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors](https://mlsanity.com/paper/2512.10835/) (2025)
- [CA2: Code-Aware Agent for Automated Game Testing](https://mlsanity.com/paper/2605.13918/) (2026)
- [Reward-Adaptive Iterative Discovery: A Case Study on Automated Game Testing for NHL26](https://mlsanity.com/paper/2607.07498/) (2026)

## Abstract (arXiv)

As modern games continue growing both in size and complexity, it has become more challenging to ensure that all the relevant content is tested and that any potential issue is properly identified and fixed. Attempting to maximize testing coverage using only human participants, however, results in a tedious and hard to orchestrate process which normally slows down the development cycle. Complementing playtesting via autonomous agents has shown great promise accelerating and simplifying this process. This paper addresses the problem of automatically exploring and testing a given scenario using reinforcement learning agents trained to maximize game state coverage. Each of these agents is rewarded based on the novelty of its actions, thus encouraging a curious and exploratory behaviour on a complex 3D scenario where previously proposed exploration techniques perform poorly. The curious agents are able to learn the complex navigation mechanics required to reach the different areas around the map, thus providing the necessary data to identify potential issues. Moreover, the paper also explores different visualization strategies and evaluates how to make better use of the collected data to drive design decisions and to recognize possible problems and oversights.