mlsanityData to 7 Oct 2026

Mastering Atari with Discrete World Models

arXiv 2010.02193 ↗First posted 5 Oct 2020cs.LG cs.AI stat.MLcited by 990

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Intelligent agents need to generalize from past experience to achieve goals in complex environments. World models facilitate such generalization and allow learning behaviors from imagined outcomes to increase sample-efficiency. While learning world models from image inputs has recently become feasible for some tasks, modeling Atari games accurately enough to derive successful behaviors has remained an open challenge for many years. We introduce DreamerV2, a reinforcement learning agent that learns behaviors purely from predictions in the compact latent space of a powerful world model. The world model uses discrete representations and is trained separately from the policy. DreamerV2 constitutes the first agent that achieves human-level performance on the Atari benchmark of 55 tasks by learning behaviors inside a separately trained world model. With the same computational budget and wall-clock time, Dreamer V2 reaches 200M frames and surpasses the final performance of the top single-GPU agents IQN and Rainbow. DreamerV2 is also applicable to tasks with continuous actions, where it learns an accurate world model of a complex humanoid robot and solves stand-up and walking from only pixel inputs.

Key facts. Cited by 990 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 11 of its 81 references that are in the map. First posted on arXiv 5 Oct 2020.

Builds on

11 of 81 references.

Closest ancestors · 9

Same direction, newest first

  1. 2020Planning to Explore via Self-Supervised World Models12 May
  2. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  3. 2019Exploring Model-based Planning with Policy Networks20 Jun
  4. 2019Deep Neuroevolution of Recurrent and Discrete World Models28 Apr
  5. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May
  6. 2018World Models27 Mar
  7. 2018Model-Ensemble Trust-Region Policy Optimization28 Feb
  8. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning8 Aug
  9. 2017Value Prediction Network11 Jul

Other references · 2

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 76; 2022: 114; 2023: 172; 2024: 188; 2025: 215; 2026: 223.

990 papers.

Sort:

By area:

  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  2. 2023Mastering Diverse Domains through World Modelscited by 1,193
  3. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  4. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  5. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  6. 2024Genie: Generative Interactive Environmentscited by 706
  7. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  8. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  9. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  10. 2023Learning Interactive Real-World Simulatorscited by 440
  11. 2026World Action Models are Zero-shot Policiescited by 411
  12. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
38 more
  1. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  2. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  3. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  4. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  5. 2022Transformers are Sample-Efficient World Modelscited by 317
  6. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  7. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  8. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  9. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  10. 2024Navigation World Modelscited by 262
  11. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 258
  12. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  13. 2021Mastering Atari Games with Limited Datacited by 235
  14. 2022Multi-Game Decision Transformerscited by 212
  15. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  16. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 193
  17. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
  18. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  19. 2022Masked World Models for Visual Controlcited by 169
  20. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  21. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  22. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 152
  23. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  24. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
  25. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  26. 2023Structured World Models from Human Videoscited by 144
  27. 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agentscited by 144
  28. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  29. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  30. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 137
  31. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 137
  32. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
  33. 2021Discovering and Achieving Goals via World Modelscited by 131
  34. 2025Training Agents Inside of Scalable World Modelscited by 122
  35. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  36. 2024Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigationcited by 116
  37. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  38. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111

50 most cited of 990 All 990 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving Weak World Models Behind Strong Agents in Atari Pong
  2. 2024Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining
  3. 2025Learning To Explore With Predictive World Model Via Self-Supervised Learning
  4. 2024Atari-GPT: Benchmarking Multimodal Large Language Models as Low-Level Policies in Atari Games
  5. 2026DashVMC: Real-Time Discrete World Model Control in Geometry Dash
  6. 2025Zero-shot World Models via Search in Memory
  7. 2025DreamerV3-XP: Optimizing exploration through uncertainty estimation
  8. 2025SIMA 2: A Generalist Embodied Agent for Virtual Worlds