mlsanityData to 7 Oct 2026

Mastering Diverse Domains through World Models

arXiv 2301.04104 ↗First posted 10 Jan 2023cs.AI cs.LG stat.MLcited by 1,193

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Developing a general algorithm that learns to solve tasks across a wide range of applications has been a fundamental challenge in artificial intelligence. Although current reinforcement learning algorithms can be readily applied to tasks similar to what they have been developed for, configuring them for new application domains requires significant human expertise and experimentation. We present DreamerV3, a general algorithm that outperforms specialized methods across over 150 diverse tasks, with a single configuration. Dreamer learns a model of the environment and improves its behavior by imagining future scenarios. Robustness techniques based on normalization, balancing, and transformations enable stable learning across domains. Applied out of the box, Dreamer is the first algorithm to collect diamonds in Minecraft from scratch without human data or curricula. This achievement has been posed as a significant challenge in artificial intelligence that requires exploring farsighted strategies from pixels and sparse rewards in an open world. Our work allows solving challenging control problems without extensive experimentation, making reinforcement learning broadly applicable.

Key facts. Cited by 1,193 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 68 of its 90 references that are in the map. First posted on arXiv 10 Jan 2023.

Builds on

68 of 90 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Transformers are Sample-Efficient World Models1 Sep
  2. 2021Mastering Atari Games with Limited Data30 Oct
  3. 2020Mastering Atari with Discrete World Models5 Oct
  4. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  5. 2018World Models27 Mar

Other references · 63

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2016Layer Normalizationcited by 5,579
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  7. 2015Continuous control with deep reinforcement learningcited by 3,279
  8. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  9. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  10. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  11. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  12. 2015Prioritized Experience Replaycited by 1,070
38 more
  1. 2018DeepMind Control Suitecited by 935
  2. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  3. 2022A Generalist Agentcited by 802
  4. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  5. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  6. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  7. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  8. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  9. 2016Deep Exploration via Bootstrapped DQNcited by 518
  10. 2019Model-Based Reinforcement Learning for Ataricited by 495
  11. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  12. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  13. 2020Reinforcement Learning with Augmented Datacited by 452
  14. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  15. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  16. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
  17. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  18. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  19. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  20. 2018Maximum a Posteriori Policy Optimisationcited by 318
  21. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  22. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  23. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  24. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  25. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  26. 2018Distributed Distributional Deterministic Policy Gradientscited by 195
  27. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 193
  28. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  29. 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 157
  30. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 149
  31. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  32. 2019Unsupervised State Representation Learning in Ataricited by 144
  33. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 138
  34. 2019Behaviour Suite for Reinforcement Learningcited by 129
  35. 2020Phasic Policy Gradientcited by 121
  36. 2022Deep Hierarchical Planning from Pixelscited by 96
  37. 2018Taming VAEscited by 85
  38. 2018Improving Regression Performance with Distributional Lossescited by 66

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 137; 2024: 232; 2025: 321; 2026: 503.

1,193 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  2. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  3. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  4. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  5. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  6. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  7. 2023Learning Interactive Real-World Simulatorscited by 440
  8. 2026World Action Models are Zero-shot Policiescited by 411
  9. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  10. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  11. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  12. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
38 more
  1. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
  2. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  3. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  4. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  5. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  6. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 264
  7. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  8. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 258
  9. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  10. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  11. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
  12. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  13. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
  14. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  15. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 150
  16. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  17. 2026Advancing Open-source World Modelscited by 144
  18. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  19. 2023Vision-Language Models as Success Detectorscited by 138
  20. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
  21. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
  22. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
  23. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 123
  24. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 117
  25. 2024Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigationcited by 116
  26. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  27. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 106
  28. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusioncited by 105
  29. 2025Agentic Large Language Models, a surveycited by 105
  30. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 104
  31. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 101
  32. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 95
  33. 2024IRASim: A Fine-Grained World Model for Robot Manipulationcited by 93
  34. 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Drivingcited by 89
  35. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  36. 2023STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learningcited by 88
  37. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokenscited by 87
  38. 2024World Models for Autonomous Driving: An Initial Surveycited by 79

50 most cited of 1,193 All 1,193 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Training Agents Inside of Scalable World Models
  2. 2023Reward-Free Curricula for Training Robust World Models
  3. 2022DayDreamer: World Models for Physical Robot Learning
  4. 2021Dropout's Dream Land: Generalization from Learned Simulators to Reality
  5. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations
  6. 2026Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidance
  7. 2025Zero-shot World Models via Search in Memory
  8. 2022DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction