mlsanityData to 7 Oct 2026

Towards Playing Full MOBA Games with Deep Reinforcement Learning

arXiv 2011.12692 ↗First posted 25 Nov 2020cs.AI cs.LGcited by 78

Not on the map. Why?

Abstract (arXiv)

MOBA games, e.g., Honor of Kings, League of Legends, and Dota 2, pose grand challenges to AI systems such as multi-agent, enormous state-action space, complex action control, etc. Developing AI for playing MOBA games has raised much attention accordingly. However, existing work falls short in handling the raw game complexity caused by the explosion of agent combinations, i.e., lineups, when expanding the hero pool in case that OpenAI's Dota AI limits the play to a pool of only 17 heroes. As a result, full MOBA games without restrictions are far from being mastered by any existing AI system. In this paper, we propose a MOBA AI learning paradigm that methodologically enables playing full MOBA games with deep reinforcement learning. Specifically, we develop a combination of novel and existing learning techniques, including curriculum self-play learning, policy distillation, off-policy adaption, multi-head value estimation, and Monte-Carlo tree-search, in training and playing a large pool of heroes, meanwhile addressing the scalability issue skillfully. Tested on Honor of Kings, a popular MOBA game, we show how to build superhuman AI agents that can defeat top esports players. The superiority of our AI is demonstrated by the first large-scale performance test of MOBA AI agent in the literature.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 18; 2023: 18; 2024: 14; 2025: 5; 2026: 6.

78 papers.

Sort:

By area:

  1. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 230
  2. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  3. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  4. 2022Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexitycited by 59
  5. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  6. 2024Enhance Reasoning for Large Language Models in the Game Werewolfcited by 44
  7. 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
  8. 2022Melting Pot 2.0cited by 37
  9. 2023Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Gamescited by 29
  10. 2022Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learningcited by 27
  11. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  12. 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
38 more
  1. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  2. 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 18
  3. 2021Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learningcited by 17
  4. 2021Coordinated Proximal Policy Optimizationcited by 16
  5. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 16
  6. 2021Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum Gamescited by 14
  7. 2022Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualizationcited by 13
  8. 2023Tackling Cooperative Incompatibility for Zero-Shot Human-AI Coordinationcited by 12
  9. 2022Revisiting Discrete Soft Actor-Criticcited by 11
  10. 2023Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Playcited by 10
  11. 2023SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Corescited by 10
  12. 2024Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarkscited by 10
  13. 2022Aries: Efficient Testing of Deep Neural Networks via Labeling-Free Accuracy Estimationcited by 9
  14. 2023Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximationcited by 9
  15. 2023A Versatile Multi-Agent Reinforcement Learning Benchmark for Inventory Managementcited by 9
  16. 2025A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Gamescited by 9
  17. 2022RLogist: Fast Observation Strategy on Whole-slide Images with Deep Reinforcement Learningcited by 8
  18. 2025Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Datacited by 8
  19. 2021Boosting Offline Reinforcement Learning with Residual Generative Modelingcited by 7
  20. 2021Learning Diverse Policies in MOBA Games via Macro-Goalscited by 7
  21. 2022CTDS: Centralized Teacher with Decentralized Student for Multi-Agent Reinforcement Learningcited by 7
  22. 2022Curriculum-based Asymmetric Multi-task Reinforcement Learningcited by 7
  23. 2024Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignmentcited by 7
  24. 2025BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systemscited by 7
  25. 2021Value Activation for Bias Alleviation: Generalized-activated Deep Double Deterministic Policy Gradientscited by 6
  26. 2022DraftRec: Personalized Draft Recommendation for Winning in Multi-Player Online Battle Arena Gamescited by 6
  27. 2023Paced-Curriculum Distillation with Prediction and Label Uncertainty for Image Segmentationcited by 6
  28. 2021Dealing with Non-Stationarity in MARL via Trust-Region Decompositioncited by 5
  29. 2023Towards Effective and Interpretable Human-Agent Collaboration in MOBA Games: A Communication Perspectivecited by 5
  30. 2023A Survey on Service Route and Time Prediction in Instant Delivery: Taxonomy, Progress, and Prospectscited by 5
  31. 2022Distributed Deep Reinforcement Learning: A Survey and A Multi-Player Multi-Agent Learning Toolboxcited by 4
  32. 2024Enhancing Human Experience in Human-Agent Collaboration: A Human-Centered Modeling Approach Based on Positive Human Gaincited by 4
  33. 2024Improving Sample Efficiency of Reinforcement Learning with Background Knowledge from Large Language Modelscited by 4
  34. 2022MCMARL: Parameterizing Value Function via Mixture of Categorical Distributions for Multi-Agent Reinforcement Learningcited by 3
  35. 2022A Survey of Decision Making in Adversarial Gamescited by 3
  36. 2022Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning towards Asynchronous Commercial Gamescited by 3
  37. 2023Emergent collective intelligence from massive-agent cooperation and competitioncited by 3
  38. 2024Near-Optimal Reinforcement Learning with Self-Play under Adaptivity Constraintscited by 3

50 most cited of 78 All 78 (JSON)