mlsanityData to 7 Oct 2026

Model-Based Reinforcement Learning for Atari

arXiv 1903.00374 ↗First posted 1 Mar 2019cs.LG stat.MLcited by 495

Not on the map. Why?

Cited by 140 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Model-free reinforcement learning (RL) can be used to learn effective policies for complex tasks, such as Atari games, even from image observations. However, this typically requires very large amounts of interaction -- substantially more, in fact, than a human would need to learn the same games. How can people learn so quickly? Part of the answer may be that people can learn how the game works and predict which actions will lead to desirable outcomes. In this paper, we explore how video prediction models can similarly enable agents to solve Atari games with fewer interactions than model-free methods. We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting. Our experiments evaluate SimPLe on a range of Atari games in low data regime of 100k interactions between the agent and the environment, which corresponds to two hours of real-time play. In most games SimPLe outperforms state-of-the-art model-free algorithms, in some games by over an order of magnitude.

Key facts. Cited by 495 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 5; 2021: 105; 2022: 79; 2023: 85; 2024: 93; 2025: 82; 2026: 46.

495 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2023Mastering Diverse Domains through World Modelscited by 1,193
  3. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  4. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  5. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  6. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  7. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  8. 2022Transformers are Sample-Efficient World Modelscited by 317
  9. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  10. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  11. 2023A Cookbook of Self-Supervised Learningcited by 248
  12. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
38 more
  1. 2021Mastering Atari Games with Limited Datacited by 235
  2. 2022Online Decision Transformercited by 194
  3. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  4. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
  5. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  6. 2022Masked World Models for Visual Controlcited by 169
  7. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  8. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  9. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  10. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  11. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 137
  12. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
  13. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  14. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  15. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  16. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  17. 2021APS: Active Pretraining with Successor Featurescited by 109
  18. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  19. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  20. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  21. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusioncited by 105
  22. 2021The Efficiency Misnomercited by 98
  23. 2023STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learningcited by 88
  24. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 86
  25. 2024Pandora: Towards General World Model with Natural Language Actions and Video Statescited by 83
  26. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  27. 2024World Models for Autonomous Driving: An Initial Surveycited by 79
  28. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 75
  29. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representationscited by 73
  30. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 71
  31. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  32. 2023SafeDiffuser: Safe Planning with Diffusion Probabilistic Modelscited by 69
  33. 2023Elastic Decision Transformercited by 66
  34. 2024Agent Planning with World Knowledge Modelcited by 66
  35. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  36. 2025WorldGym: World Model as An Environment for Policy Evaluationcited by 61
  37. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planningcited by 59
  38. 2021Learning Task Informed Abstractionscited by 58

50 most cited of 495 All 495 (JSON)