# A Survey of Deep Reinforcement Learning in Video Games

arXiv: https://arxiv.org/abs/1912.10944 · HTML: https://mlsanity.com/paper/1912.10944/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2019.

## Built on by (most cited first, 30 of 73)

- [A Survey on Large Language Model-Based Game Agents](https://mlsanity.com/paper/2404.02039/) (2024)
- [R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning](https://mlsanity.com/paper/2505.02835/) (2025)
- [PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithm](https://mlsanity.com/paper/2208.07914/) (2022)
- [Machine Learning in Aerodynamic Shape Optimization](https://mlsanity.com/paper/2202.07141/) (2022)
- [Adversarial Reinforcement Learning for Procedural Content Generation](https://mlsanity.com/paper/2103.04847/) (2021)
- [VideoGameBench: Can Vision-Language Models complete popular video games?](https://mlsanity.com/paper/2505.18134/) (2025)
- [Disentangling Transfer in Continual Reinforcement Learning](https://mlsanity.com/paper/2209.13900/) (2022)
- [Direct Behavior Specification via Constrained Reinforcement Learning](https://mlsanity.com/paper/2112.12228/) (2021)
- [Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey](https://mlsanity.com/paper/2110.01411/) (2021)
- [Why Do LLM Agents Fail in Exploring New Environments? A World-Modeling Perspective](https://mlsanity.com/paper/2510.15047/) (2025)
- [Learn-to-Race: A Multimodal Control Environment for Autonomous Racing](https://mlsanity.com/paper/2103.11575/) (2021)
- [Pareto Set Learning for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2501.06773/) (2025)
- [Excluding the Irrelevant: Focusing Reinforcement Learning through Continuous Action Masking](https://mlsanity.com/paper/2406.03704/) (2024)
- [Generative Models in Decision Making: A Survey](https://mlsanity.com/paper/2502.17100/) (2025)
- [Games for Artificial Intelligence Research: A Review and Perspectives](https://mlsanity.com/paper/2304.13269/) (2023)
- [Deep Reinforcement Learning for Band Selection in Hyperspectral Image Classification](https://mlsanity.com/paper/2103.08741/) (2021)
- [Geometry and convergence of natural policy gradient methods](https://mlsanity.com/paper/2211.02105/) (2022)
- [Transformers in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.05979/) (2023)
- [Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances](https://mlsanity.com/paper/2508.10316/) (2025)
- [Semantically-Aware Rewards for Open-Ended R1 Training in Free-Form Generation](https://mlsanity.com/paper/2506.15068/) (2025)
- [A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat](https://mlsanity.com/paper/2212.03830/) (2022)
- [Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning](https://mlsanity.com/paper/2103.04067/) (2021)
- [Learning to Shape Rewards using a Game of Two Partners](https://mlsanity.com/paper/2103.09159/) (2021)
- [A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games](https://mlsanity.com/paper/2509.03682/) (2025)
- [Advances in Preference-based Reinforcement Learning: A Review](https://mlsanity.com/paper/2408.11943/) (2024)
- [Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation](https://mlsanity.com/paper/2207.08379/) (2022)
- [Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies](https://mlsanity.com/paper/2403.15267/) (2024)
- [Causal-aware Large Language Models: Enhancing Decision-Making Through Learning, Adapting and Acting](https://mlsanity.com/paper/2505.24710/) (2025)
- [ROSARL: Reward-Only Safe Reinforcement Learning](https://mlsanity.com/paper/2306.00035/) (2023)
- [Grounded Curriculum Learning](https://mlsanity.com/paper/2409.19816/) (2024)

## Abstract (arXiv)

Deep reinforcement learning (DRL) has made great achievements since proposed. Generally, DRL agents receive high-dimensional inputs at each step, and make actions according to deep-neural-network-based policies. This learning mechanism updates the policy to maximize the return with an end-to-end method. In this paper, we survey the progress of DRL methods, including value-based, policy gradient, and model-based algorithms, and compare their main techniques and properties. Besides, DRL plays an important role in game artificial intelligence (AI). We also take a review of the achievements of DRL in various video games, including classical Arcade games, first-person perspective games and multi-agent real-time strategy games, from 2D to 3D, and from single-agent to multi-agent. A large number of video game AIs with DRL have achieved super-human performance, while there are still some challenges in this domain. Therefore, we also discuss some key points when applying DRL methods to this field, including exploration-exploitation, sample efficiency, generalization and transfer, multi-agent learning, imperfect information, and delayed spare rewards, as well as some research directions.