mlsanityData to 7 Oct 2026

Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

arXiv 1603.01121 ↗First posted 3 Mar 2016cs.LG cs.AI cs.GTcited by 153

Not on the map. Why?

Abstract (arXiv)

Many real-world applications can be described as large-scale games of imperfect information. To deal with these challenging domains, prior work has focused on computing Nash equilibria in a handcrafted abstraction of the domain. In this paper we introduce the first scalable end-to-end approach to learning approximate Nash equilibria without prior domain knowledge. Our method combines fictitious self-play with deep reinforcement learning. When applied to Leduc poker, Neural Fictitious Self-Play (NFSP) approached a Nash equilibrium, whereas common reinforcement learning methods diverged. In Limit Texas Holdem, a poker game of real-world scale, NFSP learnt a strategy that approached the performance of state-of-the-art, superhuman algorithms based on significant domain expertise.

Key facts. Cited by 153 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Mar 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 24; 2022: 25; 2023: 29; 2024: 25; 2025: 20; 2026: 30.

153 papers.

Sort:

By area:

  1. 2023Nash Learning from Human Feedbackcited by 220
  2. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  3. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 148
  4. 2021Collaborating with Humans without Human Datacited by 139
  5. 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4cited by 67
  6. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  7. 2022A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Gamescited by 64
  8. 2021Vector Quantized Models for Planningcited by 47
  9. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  10. 2025Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Playcited by 37
  11. 2021Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomizationcited by 35
  12. 2021XDO: A Double Oracle Algorithm for Extensive-Form Gamescited by 34
38 more
  1. 2021Independent Natural Policy Gradient Always Converges in Markov Potential Gamescited by 34
  2. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  3. 2022Scalable Deep Reinforcement Learning Algorithms for Mean Field Gamescited by 33
  4. 2021Unsupervised Visual Attention and Invariance for Reinforcement Learningcited by 30
  5. 2021Last-iterate Convergence in Extensive-Form Gamescited by 27
  6. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  7. 2022ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regretcited by 24
  8. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  9. 2025Measuring General Intelligence with Generated Gamescited by 21
  10. 2021Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Spacecited by 20
  11. 2021Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Reviewcited by 17
  12. 2022PerfectDou: Dominating DouDizhu with Perfect Information Distillationcited by 15
  13. 2023Policy Space Diversity for Non-Transitive Gamescited by 15
  14. 2023Adaptive Coordination in Social Embodied Rearrangementcited by 14
  15. 2025Adversarial Reinforcement Learning for Large Language Model Agent Safetycited by 14
  16. 2023Learning Near-Optimal Intrusion Responses Against Dynamic Attackerscited by 13
  17. 2024Automated Security Response through Online Learning with Adaptive Conjecturescited by 13
  18. 2025Learning Strategic Language Agents in the Werewolf Game with Iterative Latent Space Policy Optimizationcited by 13
  19. 2025Reevaluating Policy Gradient Methods for Imperfect-Information Gamescited by 13
  20. 2021Model-Free Online Learning in Unknown Sequential Decision Making Problems and Gamescited by 12
  21. 2024PUZZLES: A Benchmark for Neural Algorithmic Reasoningcited by 12
  22. 2022Self-Play PSRO: Toward Optimal Populations in Two-Player Zero-Sum Gamescited by 11
  23. 2024A Review of Reinforcement Learning in Financial Applicationscited by 11
  24. 2023Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Playcited by 10
  25. 2023Mastering Strategy Card Game (Hearthstone) with Improved Techniquescited by 10
  26. 2024Large Language Models as Agents in Two-Player Gamescited by 10
  27. 2024RL-CFR: Improving Action Abstraction for Imperfect Information Extensive-Form Games with Reinforcement Learningcited by 10
  28. 2022Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitionscited by 9
  29. 2023Robust Adversarial Reinforcement Learning via Bounded Rationality Curriculacited by 9
  30. 2023Learning Multi-Pursuit Evasion for Safe Targeted Navigation of Dronescited by 8
  31. 2023Scalable Learning of Intrusion Responses through Recursive Decompositioncited by 8
  32. 2021Solving Large-Scale Extensive-Form Network Security Games via Neural Fictitious Self-Playcited by 7
  33. 2022DouZero+: Improving DouDizhu AI by Opponent Modeling and Coach-guided Learningcited by 7
  34. 2023Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RLcited by 7
  35. 2024Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignmentcited by 7
  36. 2026Digital Red Queen: Adversarial Program Evolution in Core War with LLMscited by 7
  37. 2026How Far Are LLMs from Professional Poker Players? Revisiting Game-Theoretic Reasoning with Agentic Tool Usecited by 7
  38. 2023Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learningcited by 6

50 most cited of 153 All 153 (JSON)