mlsanityData to 7 Oct 2026

Mastering Complex Control in MOBA Games with Deep Reinforcement Learning

arXiv 1912.09729 ↗First posted 20 Dec 2019cs.AI cs.LGcited by 150

Not on the map. Why?

Abstract (arXiv)

We study the reinforcement learning problem of complex action control in the Multi-player Online Battle Arena (MOBA) 1v1 games. This problem involves far more complicated state and action spaces than those of traditional 1v1 games, such as Go and Atari series, which makes it very difficult to search any policies with human-level performance. In this paper, we present a deep reinforcement learning framework to tackle this problem from the perspectives of both system and algorithm. Our system is of low coupling and high scalability, which enables efficient explorations at large scale. Our algorithm includes several novel strategies, including control dependency decoupling, action mask, target attention, and dual-clip PPO, with which our proposed actor-critic network can be effectively trained in our system. Tested on the MOBA game Honor of Kings, our AI agent, called Tencent Solo, can defeat top professional human players in full 1v1 games.

Key facts. Cited by 150 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 19; 2022: 26; 2023: 21; 2024: 22; 2025: 29; 2026: 31.

150 papers.

Sort:

By area:

  1. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 78
  2. 2025Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoningcited by 75
  3. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  4. 2025MiniCPM4: Ultra-Efficient LLMs on End Devicescited by 51
  5. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  6. 2025Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVRcited by 38
  7. 2025DCPO: Dynamic Clipping Policy Optimizationcited by 38
  8. 2024Block-Attention for Efficient Prefillingcited by 34
  9. 2021JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learningcited by 28
  10. 2022Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learningcited by 27
  11. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  12. 2023Order Matters: Agent-by-agent Policy Optimizationcited by 24
38 more
  1. 2025Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weightingcited by 23
  2. 2026FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimizationcited by 23
  3. 2025Introducing LongCat-Flash-Thinking: A Technical Reportcited by 21
  4. 2025When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RLcited by 21
  5. 2024POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfindingcited by 20
  6. 2022You May Not Need Ratio Clipping in PPOcited by 19
  7. 2022Towards Applicable Reinforcement Learning: Improving the Generalization and Sample Efficiency with Policy Ensemblecited by 18
  8. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  9. 2022Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospectscited by 17
  10. 2023PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clippingcited by 17
  11. 2024Excluding the Irrelevant: Focusing Reinforcement Learning through Continuous Action Maskingcited by 17
  12. 2021Coordinated Proximal Policy Optimizationcited by 16
  13. 2023Visual Imitation Learning with Patch Rewardscited by 16
  14. 2020Benchmarking End-to-End Behavioural Cloning on Video Gamescited by 15
  15. 2022PerfectDou: Dominating DouDizhu with Perfect Information Distillationcited by 15
  16. 2022Spike Calibration: Fast and Accurate Conversion of Spiking Neural Network for Object Detection and Segmentationcited by 15
  17. 2021DQ-GAT: Towards Safe and Efficient Autonomous Driving with Deep Q-Learning and Graph Attention Networkscited by 13
  18. 2022Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualizationcited by 13
  19. 2021Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignmentcited by 12
  20. 2022The Sufficiency of Off-Policyness and Soft Clipping: PPO is still Insufficient according to an Off-Policy Measurecited by 12
  21. 2022Flow to Control: Offline Reinforcement Learning with Lossless Primitive Discoverycited by 12
  22. 2022Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RLcited by 11
  23. 2022Revisiting Discrete Soft Actor-Criticcited by 11
  24. 2025CPGD: Toward Stable Rule-based Reinforcement Learning for Language Modelscited by 11
  25. 2025TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedbackcited by 11
  26. 2024Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarkscited by 10
  27. 2023On the Efficacy of 3D Point Cloud Reinforcement Learningcited by 9
  28. 2024Efficient Multi-agent Reinforcement Learning by Planningcited by 9
  29. 2025Agents Play Thousands of 3D Video Gamescited by 9
  30. 2022Scenario-Assisted Deep Reinforcement Learningcited by 8
  31. 2022RLogist: Fast Observation Strategy on Whole-slide Images with Deep Reinforcement Learningcited by 8
  32. 2023Not All Tasks Are Equally Difficult: Multi-Task Deep Reinforcement Learning with Dynamic Depth Routingcited by 8
  33. 2024Affordable Generative Agentscited by 8
  34. 2021Boosting Offline Reinforcement Learning with Residual Generative Modelingcited by 7
  35. 2021Learning Diverse Policies in MOBA Games via Macro-Goalscited by 7
  36. 2022Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systemscited by 7
  37. 2022CTDS: Centralized Teacher with Decentralized Student for Multi-Agent Reinforcement Learningcited by 7
  38. 2022DouZero+: Improving DouDizhu AI by Opponent Modeling and Coach-guided Learningcited by 7

50 most cited of 150 All 150 (JSON)