mlsanityData to 7 Oct 2026

Dream to Control: Learning Behaviors by Latent Imagination

arXiv 1912.01603 ↗First posted 3 Dec 2019cs.LG cs.AI cs.ROcited by 1,598

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Learned world models summarize an agent's experience to facilitate learning complex behaviors. While learning world models from high-dimensional sensory inputs is becoming feasible through deep learning, there are many potential ways for deriving behaviors from them. We present Dreamer, a reinforcement learning agent that solves long-horizon tasks from images purely by latent imagination. We efficiently learn behaviors by propagating analytic gradients of learned state values back through trajectories imagined in the compact state space of a learned world model. On 20 challenging visual control tasks, Dreamer exceeds existing approaches in data-efficiency, computation time, and final performance.

Key facts. Cited by 1,598 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 7 of its 68 references that are in the map. First posted on arXiv 3 Dec 2019.

Builds on

7 of 68 references.

Closest ancestors · 5

Same direction, newest first

  1. 2019Exploring Model-based Planning with Policy Networks20 Jun
  2. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May
  3. 2018World Models27 Mar
  4. 2018Model-Ensemble Trust-Region Policy Optimization28 Feb
  5. 2017Value Prediction Network11 Jul

Other references · 2

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 7; 2021: 144; 2022: 181; 2023: 201; 2024: 258; 2025: 318; 2026: 489.

1,598 papers.

Sort:

By area:

  1. 2023Mastering Diverse Domains through World Modelscited by 1,193
  2. 2020Mastering Atari with Discrete World Modelscited by 990
  3. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  4. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  5. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  6. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  7. 2024Genie: Generative Interactive Environmentscited by 706
  8. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  9. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  10. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  11. 2026World Action Models are Zero-shot Policiescited by 411
  12. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
38 more
  1. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  2. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  3. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  4. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  5. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  6. 2022Transformers are Sample-Efficient World Modelscited by 317
  7. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  8. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  9. 2024Diffusion Models Are Real-Time Game Enginescited by 282
  10. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  11. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  12. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 264
  13. 2024Navigation World Modelscited by 262
  14. 2022Transformers Learn Shortcuts to Automatacited by 260
  15. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 258
  16. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  17. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  18. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  19. 2021Mastering Atari Games with Limited Datacited by 235
  20. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  21. 2022Multi-Game Decision Transformerscited by 212
  22. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  23. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  24. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
  25. 2025World Simulation with Video Foundation Models for Physical AIcited by 183
  26. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
  27. 2021Reinforcement Learning with Prototypical Representationscited by 179
  28. 2022Masked World Models for Visual Controlcited by 169
  29. 2023Language-Driven Representation Learning for Roboticscited by 163
  30. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  31. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  32. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  33. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 152
  34. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
  35. 2023Structured World Models from Human Videoscited by 144
  36. 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agentscited by 144
  37. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  38. 2023Getting aligned on representational alignmentcited by 142

50 most cited of 1,598 All 1,598 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2018Learning Real-World Robot Policies by Dreaming
  2. 2026DreamFormer: Dream Imitation with a Transformer World Model for Language-Conditioned Robotic Manipulation
  3. 2025Multimodal Dreaming: A Global Workspace Approach to World Model-Based Reinforcement Learning
  4. 2020Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning
  5. 2026GIRL: Generative Imagination Reinforcement Learning via Information-Theoretic Hallucination Control
  6. 2022Towards biologically plausible Dreaming and Planning in recurrent spiking networks
  7. 2026WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models
  8. 2026LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning