# Reinforcement Learning for High-Level Strategic Control in Tower Defense Games

arXiv: https://arxiv.org/abs/2406.07980 · HTML: https://mlsanity.com/paper/2406.07980/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 23 references that are in the map.
- First posted on arXiv 12 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Counter-Strike Deathmatch with Large-Scale Behavioural Cloning](https://mlsanity.com/paper/2104.04258/) (2021)
- [Augmenting Automated Game Testing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.15819/) (2021)
- [On Reinforcement Learning for Full-length Game of StarCraft](https://mlsanity.com/paper/1809.09095/) (2018)
- [Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learning](https://mlsanity.com/paper/2208.07811/) (2022)
- [Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Games](https://mlsanity.com/paper/2307.11105/) (2023)
- [Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Games](https://mlsanity.com/paper/2303.02160/) (2023)
- [Automated Playtesting of Matching Tile Games](https://mlsanity.com/paper/1907.06570/) (2019)
- [Towards Automated Video Game Testing: Still a Long Way to Go](https://mlsanity.com/paper/2202.12777/) (2022)
- [Statistical Modelling of Level Difficulty in Puzzle Games](https://mlsanity.com/paper/2107.03305/) (2021)
- [Preference-conditioned Pixel-based AI Agent For Game Testing](https://mlsanity.com/paper/2308.09289/) (2023)

## Built on by (most cited first, 3 of 3)

- [TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents](https://mlsanity.com/paper/2601.05899/) (2026)
- [SOPE: Stabilizing Off-Policy Evaluation for Online RL with Prior Data](https://mlsanity.com/paper/2605.05863/) (2026)
- [Hierarchical Control in Multi-Agent Games: LLM-based Planning and RL Execution](https://mlsanity.com/paper/2606.20014/) (2026)

## Abstract (arXiv)

In strategy games, one of the most important aspects of game design is maintaining a sense of challenge for players. Many mobile titles feature quick gameplay loops that allow players to progress steadily, requiring an abundance of levels and puzzles to prevent them from reaching the end too quickly. As with any content creation, testing and validation are essential to ensure engaging gameplay mechanics, enjoyable game assets, and playable levels. In this paper, we propose an automated approach that can be leveraged for gameplay testing and validation that combines traditional scripted methods with reinforcement learning, reaping the benefits of both approaches while adapting to new situations similarly to how a human player would. We test our solution on a popular tower defense game, Plants vs. Zombies. The results show that combining a learned approach, such as reinforcement learning, with a scripted AI produces a higher-performing and more robust agent than using only heuristic AI, achieving a 57.12% success rate compared to 47.95% in a set of 40 levels. Moreover, the results demonstrate the difficulty of training a general agent for this type of puzzle-like game.