mlsanityData to 14 Oct 2026

A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games

arXiv 2509.03682 ↗First posted 3 Sep 2025cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Recent advancements in multi-agent reinforcement learning (MARL) have demonstrated its application potential in modern games. Beginning with foundational work and progressing to landmark achievements such as AlphaStar in StarCraft II and OpenAI Five in Dota 2, MARL has proven capable of achieving superhuman performance across diverse game environments through techniques like self-play, supervised learning, and deep reinforcement learning. With its growing impact, a comprehensive review has become increasingly important in this field. This paper aims to provide a thorough examination of MARL's application from turn-based two-agent games to real-time multi-agent video games including popular genres such as Sports games, First-Person Shooter (FPS) games, Real-Time Strategy (RTS) games and Multiplayer Online Battle Arena (MOBA) games. We further analyze critical challenges posed by MARL in video games, including nonstationary, partial observability, sparse rewards, team coordination, and scalability, and highlight successful implementations in games like Rocket League, Minecraft, Quake III Arena, StarCraft II, Dota 2, Honor of Kings, etc. This paper offers insights into MARL in video game AI systems, proposes a novel method to estimate game complexity, and suggests future research directions to advance MARL and its applications in game development, inspiring further innovation in this rapidly evolving field.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 36 of its 88 references that are in the map. First posted on arXiv 3 Sep 2025.

Builds on

36 of 88 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  8. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  9. 2019The StarCraft Multi-Agent Challengecited by 597
  10. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  11. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  12. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
24 more
  1. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 274
  2. 2018Distributed Prioritized Experience Replaycited by 269
  3. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
  4. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
  5. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
  6. 2018Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applicationscited by 127
  7. 2017Behavior Trees in Robotics and AI: An Introductioncited by 107
  8. 2017Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Gamescited by 96
  9. 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
  10. 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78
  11. 2019A Survey of Deep Reinforcement Learning in Video Gamescited by 73
  12. 2020Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learningcited by 65
  13. 2018ViZDoom Competitions: Playing Doom from Pixelscited by 51
  14. 2017Deep Learning for Video Game Playingcited by 30
  15. 2020Deep Reinforcement Learning for Navigation in AAA Video Gamescited by 27
  16. 2019The Multi-Agent Reinforcement Learning in Malm"O (MARL"O) Competitioncited by 14
  17. 2019Creating Pro-Level AI for a Real-Time Fighting Game Using Deep Reinforcement Learningcited by 14
  18. 2018Hierarchical Macro Strategy Model for MOBA Game AIcited by 12
  19. 2023Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Gamescited by 12
  20. 2020Reinforcement Learning Agents for Ubisoft's Roller Championscited by 6
  21. 2018The Art of Drafting: A Team-Oriented Hero Recommendation System for Multiplayer Online Battle Arena Gamescited by 4
  22. 2021Expert Human-Level Driving in Gran Turismo Sport Using Deep Reinforcement Learning with Image-based Representationcited by 3
  23. 2022On the Verge of Solving Rocket League using Deep Reinforcement Learning and Sim-to-sim Transfercited by 2
  24. 2023Lucy-SKG: Learning to Play Rocket League Efficiently Using Deep Reinforcement Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 5.

9 papers.

  1. 2025Latent Collaboration in Multi-Agent Systemscited by 73
  2. 2025EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systemscited by 37
  3. 2025MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learningcited by 25
  4. 2026MASPO: Joint Prompt Optimization for LLM-based Multi-Agent Systemscited by 7
  5. 2025PolicyEvolve: Evolving Programmatic Policies by LLMs for multi-player games via Population-Based Trainingcited by 1
  6. 2026Integrating Lagrangian Neural Networks into the Dyna Framework for Reinforcement Learningcited by 1
  7. 2026Learning to Play Blackjack: A Curriculum Learning Perspectivecited by 0
  8. 2026Multi-objective Reinforcement Learning With Augmented States Requires Rewards After Deploymentcited by 0
  9. 2026Chess on Ice: Curling Tactical Decision-Making via Backward Induction and Deep Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning
  2. 2024Game Theory and Multi-Agent Reinforcement Learning : From Nash Equilibria to Evolutionary Dynamics
  3. 2026MARL-BattleGrounds: A Novel JAX-native Benchmark for Heterogeneous and Competitive Multi-agent RL
  4. 2025HLSMAC: A New StarCraft Multi-Agent Challenge for High-Level Strategic Decision-Making
  5. 2024A Meta-Game Evaluation Framework for Deep Multiagent Reinforcement Learning ↗
  6. 2019Hierarchical Reinforcement Learning for Multi-agent MOBA Game
  7. 2024A Survey on Self-play Methods in Reinforcement Learning
  8. 2025StarCraft+: Benchmarking Multi-agent Algorithms in Adversary Paradigm