# "It's Unwieldy and It Takes a Lot of Time." Challenges and Opportunities for Creating Agents in Commercial Games

arXiv: https://arxiv.org/abs/2009.00541 · HTML: https://mlsanity.com/paper/2009.00541/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2020.

## Built on by (most cited first, 17 of 17)

- [Direct Behavior Specification via Constrained Reinforcement Learning](https://mlsanity.com/paper/2112.12228/) (2021)
- [Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learning](https://mlsanity.com/paper/2208.07811/) (2022)
- [Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Games](https://mlsanity.com/paper/2303.02160/) (2023)
- [Visual Encoders for Data-Efficient Imitation Learning in Modern Video Games](https://mlsanity.com/paper/2312.02312/) (2023)
- [Policy Fusion for Adaptive and Customizable Reinforcement Learning Agents](https://mlsanity.com/paper/2104.10610/) (2021)
- [Deep Reinforcement Learning for Multi-Agent Interaction](https://mlsanity.com/paper/2208.01769/) (2022)
- [Procedural Content Generation using Behavior Trees (PCGBT)](https://mlsanity.com/paper/2107.06638/) (2021)
- [Player Modeling using Behavioral Signals in Competitive Online Games](https://mlsanity.com/paper/2112.04379/) (2021)
- [Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approach](https://mlsanity.com/paper/2510.23216/) (2025)
- [Empirically Evaluating Creative Arc Negotiation for Improvisational Decision-making](https://mlsanity.com/paper/2106.02921/) (2021)
- [Improving Sample Efficiency in Multi-Agent Reinforcement Learning for Simulated Football Games via Exploration](https://mlsanity.com/paper/2503.13077/) (2025)
- [Path Generation and Evaluation in Video Games: A Nonparametric Statistical Approach](https://mlsanity.com/paper/2506.03522/) (2025)
- [Self-correcting Reward Shaping via Language Models for Reinforcement Learning Agents in Games](https://mlsanity.com/paper/2506.23626/) (2025)
- [A Research Agenda for Usability and Generalisation in Reinforcement Learning](https://mlsanity.com/paper/2412.16970/) (2024)
- [TROFI: Trajectory-Ranked Offline Inverse Reinforcement Learning](https://mlsanity.com/paper/2506.22008/) (2025)
- [Combining Reinforcement Learning and Behavior Trees for NPCs in Video Games with AMD Schola](https://mlsanity.com/paper/2510.14154/) (2025)
- [Augmenting Game AI with Deep Reinforcement Learning](https://mlsanity.com/paper/2606.20210/) (2026)

## Abstract (arXiv)

Game agents such as opponents, non-player characters, and teammates are central to player experiences in many modern games. As the landscape of AI techniques used in the games industry evolves to adopt machine learning (ML) more widely, it is vital that the research community learn from the best practices cultivated within the industry over decades creating agents. However, although commercial game agent creation pipelines are more mature than those based on ML, opportunities for improvement still abound. As a foundation for shared progress identifying research opportunities between researchers and practitioners, we interviewed seventeen game agent creators from AAA studios, indie studios, and industrial research labs about the challenges they experienced with their professional workflows. Our study revealed several open challenges ranging from design to implementation and evaluation. We compare with literature from the research community that address the challenges identified and conclude by highlighting promising directions for future research supporting agent creation in the games industry.