mlsanityData to 14 Oct 2026

Recurrent World Models Facilitate Policy Evolution

arXiv 1809.01999 ↗First posted 4 Sep 2018cs.LG stat.MLcited by 785

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

A generative recurrent neural network is quickly trained in an unsupervised manner to model popular reinforcement learning environments through compressed spatio-temporal representations. The world model's extracted features are fed into compact and simple policies trained by evolution, achieving state of the art results in various environments. We also train our agent entirely inside of an environment generated by its own internal world model, and transfer this policy back into the actual environment. Interactive version of paper at https://worldmodels.github.io

Key facts. Cited by 785 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 2 of its 93 references that are in the map. First posted on arXiv 4 Sep 2018.

Builds on

2 of 93 references.

Closest ancestors · 2

Same direction, newest first

  1. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning8 Aug
  2. 2015On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models30 Nov

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 7; 2020: 9; 2021: 78; 2022: 80; 2023: 100; 2024: 98; 2025: 175; 2026: 238.

785 papers.

Sort:

By area:

  1. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  2. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  3. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  4. 2024Genie: Generative Interactive Environmentscited by 747
  5. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  6. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  7. 2021Implicit Behavioral Cloningcited by 452
  8. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 449
  9. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  10. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  11. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
  12. 2022Transformers are Sample-Efficient World Modelscited by 328
38 more
  1. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 319
  2. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  3. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 273
  4. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 261
  5. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 213
  6. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 212
  7. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 191
  8. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 182
  9. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 168
  10. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  11. 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agentscited by 154
  12. 2022Model-Based Imitation Learning for Urban Drivingcited by 150
  13. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 148
  14. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 146
  15. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 139
  16. 2025Video World Models with Long-term Spatial Memorycited by 136
  17. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 135
  18. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  19. 2023Language Models Meet World Models: Embodied Experiences Enhance Language Modelscited by 124
  20. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
  21. 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
  22. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 115
  23. 2021Learning to drive from a world on railscited by 114
  24. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 114
  25. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  26. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 108
  27. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 107
  28. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusioncited by 106
  29. 2023ADriver-I: A General World Model for Autonomous Drivingcited by 104
  30. 2025TesserAct: Learning 4D Embodied World Modelscited by 94
  31. 2019Exploring Model-based Planning with Policy Networkscited by 90
  32. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokenscited by 89
  33. 2024Pandora: Towards General World Model with Natural Language Actions and Video Statescited by 86
  34. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 85
  35. 2025Agent Learning via Early Experiencecited by 85
  36. 2024World Models for Autonomous Driving: An Initial Surveycited by 80
  37. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 80
  38. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79

50 most cited of 785 All 785 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2018World Models
  2. 2020Evolutionary Planning in Latent Space
  3. 2025Language-conditioned world model improves policy generalization by reading environmental descriptions
  4. 2024Revisiting Generative Policies: A Simpler Reinforcement Learning Algorithmic Perspective
  5. 2024PWM: Policy Learning with Multi-Task World Models
  6. 2026Policy and World Modeling Co-Training for Language Agents
  7. 2026Investigating Action Encodings in Recurrent Neural Networks in Reinforcement Learning
  8. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models