mlsanityData to 7 Oct 2026

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

arXiv 1911.08265 ↗First posted 19 Nov 2019cs.LG stat.MLcited by 1,327

Not on the map. Why?

Cited by 234 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Constructing agents with planning capabilities has long been one of the main challenges in the pursuit of artificial intelligence. Tree-based planning methods have enjoyed huge success in challenging domains, such as chess and Go, where a perfect simulator is available. However, in real-world problems the dynamics governing the environment are often complex and unknown. In this work we present the MuZero algorithm which, by combining a tree-based search with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, without any knowledge of their underlying dynamics. MuZero learns a model that, when applied iteratively, predicts the quantities most directly relevant to planning: the reward, the action-selection policy, and the value function. When evaluated on 57 different Atari games - the canonical video game environment for testing AI techniques, in which model-based planning approaches have historically struggled - our new algorithm achieved a new state of the art. When evaluated on Go, chess and shogi, without any knowledge of the game rules, MuZero matched the superhuman performance of the AlphaZero algorithm that was supplied with the game rules.

Key facts. Cited by 1,327 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 3; 2021: 189; 2022: 208; 2023: 233; 2024: 236; 2025: 229; 2026: 229.

1,327 papers.

Sort:

By area:

  1. 2023Mastering Diverse Domains through World Modelscited by 1,193
  2. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  3. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  4. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  5. 2022A Generalist Agentcited by 802
  6. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  7. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  8. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  9. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  10. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  11. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  12. 2022Transformers are Sample-Efficient World Modelscited by 317
38 more
  1. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  2. 2023Tool Learning with Foundation Modelscited by 302
  3. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 291
  4. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  5. 2023A Cookbook of Self-Supervised Learningcited by 248
  6. 2021Mastering Atari Games with Limited Datacited by 235
  7. 2022Multi-Game Decision Transformerscited by 212
  8. 2021Towards Causal Representation Learningcited by 207
  9. 2022Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpointscited by 207
  10. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  11. 2024DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Searchcited by 189
  12. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 186
  13. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  14. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  15. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  16. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  17. 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 158
  18. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 153
  19. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  20. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  21. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  22. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  23. 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agentscited by 144
  24. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  25. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  26. 2024Stream of Search (SoS): Learning to Search in Languagecited by 141
  27. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  28. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 137
  29. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
  30. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
  31. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
  32. 2025Training Agents Inside of Scalable World Modelscited by 122
  33. 2021Learning to drive from a world on railscited by 114
  34. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  35. 20213D Neural Scene Representations for Visuomotor Controlcited by 110
  36. 2021APS: Active Pretraining with Successor Featurescited by 109
  37. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 106
  38. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105

50 most cited of 1,327 All 1,327 (JSON)