# Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learning

arXiv: https://arxiv.org/abs/2208.07811 · HTML: https://mlsanity.com/paper/2208.07811/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 30 of its 56 references that are in the map.
- First posted on arXiv 15 Aug 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 30 of 30)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [One-Shot Imitation Learning](https://mlsanity.com/paper/1703.07326/) (2017)
- [A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models](https://mlsanity.com/paper/1611.03852/) (2016)
- [S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learning](https://mlsanity.com/paper/2103.06326/) (2021)
- [Coordinated Multi-Agent Imitation Learning](https://mlsanity.com/paper/1703.03121/) (2017)
- [Augmenting Automated Game Testing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.15819/) (2021)
- [Hierarchical Few-Shot Imitation with Skill Transition Models](https://mlsanity.com/paper/2107.08981/) (2021)
- [Adversarial Reinforcement Learning for Procedural Content Generation](https://mlsanity.com/paper/2103.04847/) (2021)
- [Direct Behavior Specification via Constrained Reinforcement Learning](https://mlsanity.com/paper/2112.12228/) (2021)
- [Automated Playtesting with Procedural Personas through MCTS with Evolved Heuristics](https://mlsanity.com/paper/1802.06881/) (2018)
- [Improving Playtesting Coverage via Curiosity Driven Reinforcement Learning Agents](https://mlsanity.com/paper/2103.13798/) (2021)
- [Deep Reinforcement Learning for Navigation in AAA Video Games](https://mlsanity.com/paper/2011.04764/) (2020)
- [Inverse reinforcement learning for video games](https://mlsanity.com/paper/1810.10593/) (2018)
- [Imitation Learning with Concurrent Actions in 3D Games](https://mlsanity.com/paper/1803.05402/) (2018)
- ["It's Unwieldy and It Takes a Lot of Time." Challenges and Opportunities for Creating Agents in Commercial Games](https://mlsanity.com/paper/2009.00541/) (2020)
- [Winning Isn't Everything: Enhancing Game Development with Intelligent Agents](https://mlsanity.com/paper/1903.10545/) (2019)
- [Automated Playtesting of Matching Tile Games](https://mlsanity.com/paper/1907.06570/) (2019)
- [Explainable Artificial Intelligence (XAI) for Increasing User Trust in Deep Reinforcement Learning Driven Autonomous Systems](https://mlsanity.com/paper/2106.03775/) (2021)
- [CCPT: Automatic Gameplay Testing and Validation with Curiosity-Conditioned Proximal Trajectories](https://mlsanity.com/paper/2202.10057/) (2022)
- [Augmented Behavioral Cloning from Observation](https://mlsanity.com/paper/2004.13529/) (2020)
- [Towards Automated Video Game Testing: Still a Long Way to Go](https://mlsanity.com/paper/2202.12777/) (2022)
- [Policy Fusion for Adaptive and Customizable Reinforcement Learning Agents](https://mlsanity.com/paper/2104.10610/) (2021)

## Built on by (most cited first, 13 of 13)

- [Visual Encoders for Data-Efficient Imitation Learning in Modern Video Games](https://mlsanity.com/paper/2312.02312/) (2023)
- [Improving Generalization in Game Agents with Data Augmentation in Imitation Learning](https://mlsanity.com/paper/2309.12815/) (2023)
- [Generating Personas for Games with Multimodal Adversarial Imitation Learning](https://mlsanity.com/paper/2308.07598/) (2023)
- [Reinforcement Learning for High-Level Strategic Control in Tower Defense Games](https://mlsanity.com/paper/2406.07980/) (2024)
- [Real-Time Diffusion Policies for Games: Enhancing Consistency Policies with Q-Ensembles](https://mlsanity.com/paper/2503.16978/) (2025)
- [Bayesian Optimization-based Search for Agent Control in Automated Game Testing](https://mlsanity.com/paper/2508.13121/) (2025)
- [Minimax Exploiter: A Data Efficient Approach for Competitive Self-Play](https://mlsanity.com/paper/2311.17190/) (2023)
- [A Benchmark Environment for Offline Reinforcement Learning in Racing Games](https://mlsanity.com/paper/2407.09415/) (2024)
- [Self-correcting Reward Shaping via Language Models for Reinforcement Learning Agents in Games](https://mlsanity.com/paper/2506.23626/) (2025)
- [TROFI: Trajectory-Ranked Offline Inverse Reinforcement Learning](https://mlsanity.com/paper/2506.22008/) (2025)
- [VRScout: Towards Real-Time, Autonomous Testing of Virtual Reality Games](https://mlsanity.com/paper/2511.00002/) (2025)
- [ARC-RL: A Reinforcement Learning Playground Inspired by ARC Raiders](https://mlsanity.com/paper/2605.19503/) (2026)
- [Augmenting Game AI with Deep Reinforcement Learning](https://mlsanity.com/paper/2606.20210/) (2026)

## Abstract (arXiv)

In games, as in and many other domains, design validation and testing is a huge challenge as systems are growing in size and manual testing is becoming infeasible. This paper proposes a new approach to automated game validation and testing. Our method leverages a data-driven imitation learning technique, which requires little effort and time and no knowledge of machine learning or programming, that designers can use to efficiently train game testing agents. We investigate the validity of our approach through a user study with industry experts. The survey results show that our method is indeed a valid approach to game validation and that data-driven programming would be a useful aid to reducing effort and increasing quality of modern playtesting. The survey also highlights several open challenges. With the help of the most recent literature, we analyze the identified challenges and propose future research directions suitable for supporting and maximizing the utility of our approach.