# Deep Reinforcement Learning for Navigation in AAA Video Games

arXiv: https://arxiv.org/abs/2011.04764 · HTML: https://mlsanity.com/paper/2011.04764/

## Key facts

- Cited by 27 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 9 Nov 2020.

## Built on by (most cited first, 27 of 27)

- [Direct Behavior Specification via Constrained Reinforcement Learning](https://mlsanity.com/paper/2112.12228/) (2021)
- [Navigation Turing Test (NTT): Learning to Evaluate Human-Like Navigation](https://mlsanity.com/paper/2105.09637/) (2021)
- [Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learning](https://mlsanity.com/paper/2208.07811/) (2022)
- [A Survey on Reinforcement Learning Methods in Character Animation](https://mlsanity.com/paper/2203.04735/) (2022)
- [Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Games](https://mlsanity.com/paper/2307.11105/) (2023)
- [Automated Play-Testing Through RL Based Human-Like Play-Styles Generation](https://mlsanity.com/paper/2211.17188/) (2022)
- [Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Games](https://mlsanity.com/paper/2303.02160/) (2023)
- [Learning Robust Real-Time Cultural Transmission without Human Data](https://mlsanity.com/paper/2203.00715/) (2022)
- [A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games](https://mlsanity.com/paper/2509.03682/) (2025)
- [A Rubric for Human-like Agents and NeuroAI](https://mlsanity.com/paper/2212.04401/) (2022)
- [Improving Intrinsic Exploration by Creating Stationary Objectives](https://mlsanity.com/paper/2310.18144/) (2023)
- [CCPT: Automatic Gameplay Testing and Validation with Curiosity-Conditioned Proximal Trajectories](https://mlsanity.com/paper/2202.10057/) (2022)
- [Graph augmented Deep Reinforcement Learning in the GameRLand3D environment](https://mlsanity.com/paper/2112.11731/) (2021)
- [Policy Fusion for Adaptive and Customizable Reinforcement Learning Agents](https://mlsanity.com/paper/2104.10610/) (2021)
- [Godot Reinforcement Learning Agents](https://mlsanity.com/paper/2112.03636/) (2021)
- [Honey, I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL](https://mlsanity.com/paper/2102.11893/) (2021)
- [Learning Computational Efficient Bots with Costly Features](https://mlsanity.com/paper/2308.09629/) (2023)
- [Navigation with QPHIL: Quantizing Planner for Hierarchical Implicit Q-Learning](https://mlsanity.com/paper/2411.07760/) (2024)
- [Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approach](https://mlsanity.com/paper/2510.23216/) (2025)
- [WILD-SCAV: Benchmarking FPS Gaming AI on Unity3D-based Environments](https://mlsanity.com/paper/2210.09026/) (2022)
- [Path Generation and Evaluation in Video Games: A Nonparametric Statistical Approach](https://mlsanity.com/paper/2506.03522/) (2025)
- [Human-Like Navigation Behavior: A Statistical Evaluation Framework](https://mlsanity.com/paper/2203.05965/) (2022)
- [BluNF: Blueprint Neural Field](https://mlsanity.com/paper/2309.03933/) (2023)
- [Replication of Multi-agent Reinforcement Learning for the "Hide and Seek" Problem](https://mlsanity.com/paper/2310.05430/) (2023)
- [Effective Reward Specification in Deep Reinforcement Learning](https://mlsanity.com/paper/2412.07177/) (2024)
- [Combining Reinforcement Learning and Behavior Trees for NPCs in Video Games with AMD Schola](https://mlsanity.com/paper/2510.14154/) (2025)
- [Augmenting Game AI with Deep Reinforcement Learning](https://mlsanity.com/paper/2606.20210/) (2026)

## Abstract (arXiv)

In video games, non-player characters (NPCs) are used to enhance the players' experience in a variety of ways, e.g., as enemies, allies, or innocent bystanders. A crucial component of NPCs is navigation, which allows them to move from one point to another on the map. The most popular approach for NPC navigation in the video game industry is to use a navigation mesh (NavMesh), which is a graph representation of the map, with nodes and edges indicating traversable areas. Unfortunately, complex navigation abilities that extend the character's capacity for movement, e.g., grappling hooks, jetpacks, teleportation, or double-jumps, increases the complexity of the NavMesh, making it intractable in many practical scenarios. Game designers are thus constrained to only add abilities that can be handled by a NavMesh if they want to have NPC navigation. As an alternative, we propose to use Deep Reinforcement Learning (Deep RL) to learn how to navigate 3D maps using any navigation ability. We test our approach on complex 3D environments in the Unity game engine that are notably an order of magnitude larger than maps typically used in the Deep RL literature. One of these maps is directly modeled after a Ubisoft AAA game. We find that our approach performs surprisingly well, achieving at least 90% success rate on all tested scenarios. A video of our results is available at https://youtu.be/WFIf9Wwlq8M.