mlsanityData to 7 Oct 2026

Learning Latent Dynamics for Planning from Pixels

arXiv 1811.04551 ↗First posted 12 Nov 2018cs.LG cs.AI stat.MLcited by 1,221

On the map: World models → Latent predictive world models

Abstract (arXiv)

Planning has been very successful for control tasks with known environment dynamics. To leverage planning in unknown environments, the agent needs to learn the dynamics from interactions with the world. However, learning dynamics models that are accurate enough for planning has been a long-standing challenge, especially in image-based domains. We propose the Deep Planning Network (PlaNet), a purely model-based agent that learns the environment dynamics from images and chooses actions through fast online planning in latent space. To achieve high performance, the dynamics model must accurately predict the rewards ahead for multiple time steps. We approach this using a latent dynamics model with both deterministic and stochastic transition components. Moreover, we propose a multi-step variational inference objective that we name latent overshooting. Using only pixel observations, our agent solves continuous control tasks with contact dynamics, partial observability, and sparse rewards, which exceed the difficulty of tasks that were previously solved by planning with learned models. PlaNet uses substantially fewer episodes and reaches final performance close to and sometimes higher than strong model-free algorithms.

Key facts. Cited by 1,221 papers in the map (data through 7 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 5 of its 61 references that are in the map. First posted on arXiv 12 Nov 2018.

Builds on

5 of 61 references.

References · 5

  1. 2018World Modelscited by 1,444
  2. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  3. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  4. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 214
  5. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 8; 2020: 8; 2021: 159; 2022: 174; 2023: 170; 2024: 174; 2025: 213; 2026: 315.

1,221 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  3. 2023Mastering Diverse Domains through World Modelscited by 1,193
  4. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  5. 2020Mastering Atari with Discrete World Modelscited by 990
  6. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  7. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  8. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  9. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  10. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  11. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  12. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
38 more
  1. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  2. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  3. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  4. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  5. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  6. 2022Transformers are Sample-Efficient World Modelscited by 317
  7. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  8. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  9. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  10. 2021Mastering Atari Games with Limited Datacited by 235
  11. 2022Multi-Game Decision Transformerscited by 212
  12. 2019Contrastive Learning of Structured World Modelscited by 206
  13. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
  14. 2021Reinforcement Learning with Prototypical Representationscited by 179
  15. 2022Masked World Models for Visual Controlcited by 169
  16. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  17. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  18. 2023Structured World Models from Human Videoscited by 144
  19. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  20. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  21. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  22. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 137
  23. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
  24. 2022Diffusion Models for Video Prediction and Infillingcited by 134
  25. 2021Discovering and Achieving Goals via World Modelscited by 131
  26. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
  27. 2025Unified Vision-Language-Action Modelcited by 128
  28. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  29. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  30. 2025Training Agents Inside of Scalable World Modelscited by 122
  31. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  32. 2024Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigationcited by 116
  33. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  34. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  35. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  36. 20213D Neural Scene Representations for Visuomotor Controlcited by 110
  37. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  38. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101

50 most cited of 1,221 All 1,221 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
  2. 2026Latent Geometry Beyond Search: Amortizing Planning in World Models
  3. 2020Planning from Pixels using Inverse Dynamics Models
  4. 2026SAGE: Subgoal-Conditioned Action Generation for Latent World Model Planning
  5. 2026Control-Geometry Straightening for Sampling-Based Latent Planning
  6. 2024Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inference
  7. 2026LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
  8. 2025Efficient Robotic Policy Learning via Latent Space Backward Planning