mlsanityData to 14 Oct 2026

Dota 2 with Large Scale Deep Reinforcement Learning

arXiv 1912.06680 ↗First posted 13 Dec 2019cs.LG stat.MLcited by 1,103

Not on the map. Why?

Cited by 204 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

On April 13th, 2019, OpenAI Five became the first AI system to defeat the world champions at an esports game. The game of Dota 2 presents novel challenges for AI systems such as long time horizons, imperfect information, and complex, continuous state-action spaces, all challenges which will become increasingly central to more capable AI systems. OpenAI Five leveraged existing reinforcement learning techniques, scaled to learn from batches of approximately 2 million frames every 2 seconds. We developed a distributed training system and tools for continual training which allowed us to train OpenAI Five for 10 months. By defeating the Dota 2 world champion (Team OG), OpenAI Five demonstrates that self-play reinforcement learning can achieve superhuman performance on a difficult task.

Key facts. Cited by 1,103 papers in the map (data through 14 Oct 2026). First posted on arXiv 13 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 5; 2021: 188; 2022: 214; 2023: 217; 2024: 169; 2025: 182; 2026: 128.

1,103 papers.

Sort:

By area:

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  3. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  4. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  5. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  6. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  7. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  8. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  9. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  10. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  11. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
  12. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
38 more
  1. 2026Reinforcement Learning via Self-Distillationcited by 332
  2. 2022Transformers are Sample-Efficient World Modelscited by 328
  3. 2023Tool Learning with Foundation Modelscited by 302
  4. 2021Scaling Laws for Transfercited by 272
  5. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
  6. 2021Mastering Atari Games with Limited Datacited by 237
  7. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 231
  8. 2021Towards Causal Representation Learningcited by 209
  9. 2022Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpointscited by 209
  10. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  11. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  12. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  13. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  14. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  15. 2022Compute Trends Across Three Eras of Machine Learningcited by 163
  16. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 160
  17. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  18. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 156
  19. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 152
  20. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 147
  21. 2022Formal Mathematics Statement Curriculum Learningcited by 142
  22. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 139
  23. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  24. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 131
  25. 2024Cradle: Empowering Foundation Agents Towards General Computer Controlcited by 117
  26. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 116
  27. 2021Replay-Guided Adversarial Environment Designcited by 111
  28. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
  29. 2025Agentic Large Language Models, a surveycited by 108
  30. 2021B-Pref: Benchmarking Preference-Based Reinforcement Learningcited by 106
  31. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  32. 2022Is Power-Seeking AI an Existential Risk?cited by 105
  33. 2022Discovered Policy Optimisationcited by 104
  34. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 104
  35. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learningcited by 100
  36. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 99
  37. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 97
  38. 2022SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learningcited by 97

50 most cited of 1,103 All 1,103 (JSON)