# Strategies for Using Proximal Policy Optimization in Mobile Puzzle Games

arXiv: https://arxiv.org/abs/2007.01542 · HTML: https://mlsanity.com/paper/2007.01542/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2020.

## Built on by (most cited first, 6 of 6)

- [A2C is a special case of PPO](https://mlsanity.com/paper/2205.09123/) (2022)
- [Optimizing Vital Sign Monitoring in Resource-Constrained Maternal Care: An RL-Based Restless Bandit Approach](https://mlsanity.com/paper/2410.08377/) (2024)
- [Parametric-Task MAP-Elites](https://mlsanity.com/paper/2402.01275/) (2024)
- [Statistical Modelling of Level Difficulty in Puzzle Games](https://mlsanity.com/paper/2107.03305/) (2021)
- [Estimating player completion rate in mobile puzzle games using reinforcement learning](https://mlsanity.com/paper/2306.14626/) (2023)
- [Difficulty Modelling in Mobile Puzzle Games: An Empirical Study on Different Methods to Combine Player Analytics and Simulated Data](https://mlsanity.com/paper/2401.17436/) (2024)

## Abstract (arXiv)

While traditionally a labour intensive task, the testing of game content is progressively becoming more automated. Among the many directions in which this automation is taking shape, automatic play-testing is one of the most promising thanks also to advancements of many supervised and reinforcement learning (RL) algorithms. However these type of algorithms, while extremely powerful, often suffer in production environments due to issues with reliability and transparency in their training and usage. In this research work we are investigating and evaluating strategies to apply the popular RL method Proximal Policy Optimization (PPO) in a casual mobile puzzle game with a specific focus on improving its reliability in training and generalization during game playing. We have implemented and tested a number of different strategies against a real-world mobile puzzle game (Lily's Garden from Tactile Games). We isolated the conditions that lead to a failure in either training or generalization during testing and we identified a few strategies to ensure a more stable behaviour of the algorithm in this game genre.