mlsanityData to 7 Oct 2026

DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning

arXiv 2106.06135 ↗First posted 11 Jun 2021cs.AI cs.LGcited by 65

Not on the map. Why?

Abstract (arXiv)

Games are abstractions of the real world, where artificial agents learn to compete and cooperate with other agents. While significant achievements have been made in various perfect- and imperfect-information games, DouDizhu (a.k.a. Fighting the Landlord), a three-player card game, is still unsolved. DouDizhu is a very challenging domain with competition, collaboration, imperfect information, large state space, and particularly a massive set of possible actions where the legal actions vary significantly from turn to turn. Unfortunately, modern reinforcement learning algorithms mainly focus on simple and small action spaces, and not surprisingly, are shown not to make satisfactory progress in DouDizhu. In this work, we propose a conceptually simple yet effective DouDizhu AI system, namely DouZero, which enhances traditional Monte-Carlo methods with deep neural networks, action encoding, and parallel actors. Starting from scratch in a single server with four GPUs, DouZero outperformed all the existing DouDizhu AI programs in days of training and was ranked the first in the Botzone leaderboard among 344 AI agents. Through building DouZero, we show that classic Monte-Carlo methods can be made to deliver strong results in a hard domain with a complex action space. The code and an online demo are released at https://github.com/kwai/DouZero with the hope that this insight could motivate future work.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 35 of its 60 references that are in the map. First posted on arXiv 11 Jun 2021.

Builds on

35 of 60 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  6. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  8. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  9. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 265
  10. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 228
  11. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 209
  12. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 180
23 more
  1. 2020Revisiting Fundamentals of Experience Replaycited by 171
  2. 2015Deep Reinforcement Learning in Large Discrete Action Spacescited by 158
  3. 2016Deep Reinforcement Learning from Self-Play in Imperfect-Information Gamescited by 153
  4. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 150
  5. 2020Combining Deep Reinforcement Learning and Search for Imperfect-Information Gamescited by 110
  6. 2017A Deeper Look at Experience Replaycited by 104
  7. 2018Deep Counterfactual Regret Minimizationcited by 102
  8. 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78
  9. 2018Learn What Not to Learn: Action Elimination with Deep Reinforcement Learningcited by 77
  10. 2018Simple random search provides a competitive approach to reinforcement learningcited by 75
  11. 2018Solving Imperfect-Information Games via Discounted Regret Minimizationcited by 68
  12. 2018Modeling Others using Oneself in Multi-Agent Reinforcement Learningcited by 64
  13. 2020Suphx: Mastering Mahjong with Deep Reinforcement Learningcited by 55
  14. 2019Experience Replay Optimizationcited by 47
  15. 2019TorchBeast: A PyTorch Platform for Distributed RLcited by 41
  16. 2020Human-Level Performance in No-Press Diplomacy via Equilibrium Searchcited by 41
  17. 2019RLCard: A Toolkit for Reinforcement Learning in Card Gamescited by 34
  18. 2019Improving Policies via Search in Cooperative Partially Observable Gamescited by 33
  19. 2021Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environmentscited by 28
  20. 2018Generative Adversarial Self-Imitation Learningcited by 25
  21. 2020Dual Policy Distillationcited by 25
  22. 2021Simplifying Deep Reinforcement Learning via Self-Supervisioncited by 9
  23. 2019Combinational Q-Learning for Dou Di Zhucited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 15; 2023: 17; 2024: 10; 2025: 10; 2026: 8.

65 papers.

Sort:

By area:

  1. 2023Data-centric Artificial Intelligence: A Surveycited by 202
  2. 2024Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimizationcited by 93
  3. 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4cited by 67
  4. 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervisioncited by 31
  5. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  6. 2022ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regretcited by 24
  7. 2023Neurosymbolic Reinforcement Learning and Planning: A Surveycited by 24
  8. 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 18
  9. 2022AutoShard: Automated Embedding Table Sharding for Recommender Systemscited by 17
  10. 2022PerfectDou: Dominating DouDizhu with Perfect Information Distillationcited by 15
  11. 2021Variational Quantum Soft Actor-Criticcited by 14
  12. 2022DreamShard: Generalizable Embedding Table Placement for Recommender Systemscited by 14
38 more
  1. 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
  2. 2022Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RLcited by 11
  3. 2022Towards Automated Imbalanced Learning with Deep Hierarchical Reinforcement Learningcited by 9
  4. 2022Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systemscited by 7
  5. 2022DouZero+: Improving DouDizhu AI by Opponent Modeling and Coach-guided Learningcited by 7
  6. 2026Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learningcited by 7
  7. 2021AutoVideo: An Automated Video Action Recognition Systemcited by 6
  8. 2023DanZero+: Dominating the GuanDan Game through Reinforcement Learningcited by 6
  9. 2022Reinforcement Learning in Practice: Opportunities and Challengescited by 5
  10. 2022DanZero: Mastering GuanDan Game with Reinforcement Learningcited by 5
  11. 2023Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Modelscited by 5
  12. 2023An Integrated Framework Integrating Monte Carlo Tree Search and Supervised Learning for Train Timetabling Problemcited by 5
  13. 2025ORVIT: Near-Optimal Online Distributionally Robust Reinforcement Learningcited by 5
  14. 2022Distributed Deep Reinforcement Learning: A Survey and A Multi-Player Multi-Agent Learning Toolboxcited by 4
  15. 2025Can Large Language Models Master Complex Card Games?cited by 4
  16. 2022A Deep Reinforcement Learning Approach for Finding Non-Exploitable Strategies in Two-Player Atari Gamescited by 3
  17. 2023Unlabeled Imperfect Demonstrations in Adversarial Imitation Learningcited by 3
  18. 2023Imitation Learning from Purified Demonstrationscited by 3
  19. 2023JiangJun: Mastering Xiangqi by Tackling Non-Transitivity in Two-Player Zero-Sum Gamescited by 2
  20. 2023Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalizationcited by 2
  21. 2024RL-LLM-DT: An Automatic Decision Tree Generation Method Based on RL Evaluation and LLM Enhancementcited by 2
  22. 2025Cardiverse: Harnessing LLMs for Novel Card Game Prototypingcited by 2
  23. 2021Hierarchical Policy for Non-prehensile Multi-object Rearrangement with Deep Reinforcement Learning and Monte Carlo Tree Searchcited by 1
  24. 2021AI in Human-computer Gaming: Techniques, Challenges and Opportunitiescited by 1
  25. 2022Efficient Distributed Framework for Collaborative Multi-Agent Reinforcement Learningcited by 1
  26. 2022Towards Modern Card Games with Large-Scale Action Spaces Through Action Representationcited by 1
  27. 2023Towards Personalized Preprocessing Pipeline Searchcited by 1
  28. 2024Mastering the Game of Guandan with Deep Reinforcement Learning and Behavior Regulatingcited by 1
  29. 2025Empowering LLMs in Decision Games through Algorithmic Data Synthesiscited by 1
  30. 2025PolicyEvol-Agent: Evolving Policy via Environment Perception and Self-Awareness with Theory of Mindcited by 1
  31. 2025Online Robust Reinforcement Learning with General Function Approximationcited by 1
  32. 2026AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Gamescited by 1
  33. 2021On the complexity of Dark Chinese Chesscited by 0
  34. 2022Classifying Ambiguous Identities in Hidden-Role Stochastic Games with Multi-Agent Reinforcement Learningcited by 0
  35. 2023Non-zero-sum Game Control for Multi-vehicle Driving via Reinforcement Learningcited by 0
  36. 2023Policy Dispersion in Non-Markovian Environmentcited by 0
  37. 2023Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbationscited by 0
  38. 2023Accelerating Nash Equilibrium Convergence in Monte Carlo Settings Through Counterfactual Value Based Fictitious Playcited by 0

50 most cited of 65 All 65 (JSON)