mlsanityData to 7 Oct 2026

Diffusion for World Modeling: Visual Details Matter in Atari

arXiv 2405.12399 ↗First posted 20 May 2024cs.LG cs.AI cs.CVcited by 285

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

World models constitute a promising approach for training reinforcement learning agents in a safe and sample-efficient manner. Recent world models predominantly operate on sequences of discrete latent variables to model environment dynamics. However, this compression into a compact discrete representation may ignore visual details that are important for reinforcement learning. Concurrently, diffusion models have become a dominant approach for image generation, challenging well-established methods modeling discrete latents. Motivated by this paradigm shift, we introduce DIAMOND (DIffusion As a Model Of eNvironment Dreams), a reinforcement learning agent trained in a diffusion world model. We analyze the key design choices that are required to make diffusion suitable for world modeling, and demonstrate how improved visual details can lead to improved agent performance. DIAMOND achieves a mean human normalized score of 1.46 on the competitive Atari 100k benchmark; a new best for agents trained entirely within a world model. We further demonstrate that DIAMOND's diffusion world model can stand alone as an interactive neural game engine by training on static Counter-Strike: Global Offensive gameplay. To foster future research on diffusion for world modeling, we release our code, agents, videos and playable world models at https://diamond-wm.github.io.

Key facts. Cited by 285 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 68 of its 89 references that are in the map. First posted on arXiv 20 May 2024.

Builds on

68 of 89 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Policy-Guided Diffusion9 Apr
  2. 2024Diffusion World Model: Future Modeling Beyond Step-by-Step Rollout for Offline Reinforcement Learning5 Feb
  3. 2023STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning14 Oct
  4. 2023Transformer-based World Models Are Happy With 100k Interactions13 Mar
  5. 2023Mastering Diverse Domains through World Models10 Jan
  6. 2022Transformers are Sample-Efficient World Models1 Sep
  7. 2022DayDreamer: World Models for Physical Robot Learning28 Jun
  8. 2021Mastering Atari Games with Limited Data30 Oct
  9. 2020Mastering Atari with Discrete World Models5 Oct
  10. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  11. 2018Recurrent World Models Facilitate Policy Evolution4 Sep

Other references · 57

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2022Training language models to follow instructions with human feedbackcited by 15,712
  8. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  10. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
38 more
  1. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  2. 2022Scalable Diffusion Models with Transformerscited by 5,931
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  9. 2022Video Diffusion Modelscited by 2,280
  10. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  12. 2018Group Normalizationcited by 1,514
  13. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  14. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  15. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  16. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  17. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  18. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  19. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  20. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  21. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  22. 2024Genie: Generative Interactive Environmentscited by 706
  23. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  24. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  25. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  26. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  27. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  28. 2019Model-Based Reinforcement Learning for Ataricited by 495
  29. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  30. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  31. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  32. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  33. 2023Imitating Human Behaviour with Diffusion Modelscited by 260
  34. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  35. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 211
  36. 2023AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Plannerscited by 147
  37. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  38. 2015Neural GPUs Learn Algorithmscited by 118

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 119; 2026: 150.

285 papers.

Sort:

By area:

  1. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  2. 2024Navigation World Modelscited by 262
  3. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  4. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 178
  5. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
  6. 2025Flow Q-Learningcited by 162
  7. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 152
  8. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 152
  9. 2026Advancing Open-source World Modelscited by 144
  10. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 126
  11. 2024GameGen-X: Interactive Open-world Game Video Generationcited by 124
  12. 2025GameFactory: Creating New Games with Generative Interactive Videoscited by 124
38 more
  1. 2025Video World Models with Long-term Spatial Memorycited by 124
  2. 2025Training Agents Inside of Scalable World Modelscited by 122
  3. 2025Test-Time Training Done Rightcited by 121
  4. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 117
  5. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  6. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  7. 2024IRASim: A Fine-Grained World Model for Robot Manipulationcited by 93
  8. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 89
  9. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 86
  10. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
  11. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 79
  12. 2024Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharingcited by 70
  13. 2025Vid2World: Crafting Video Diffusion Models to Interactive World Modelscited by 67
  14. 2025WorldGym: World Model as An Environment for Policy Evaluationcited by 61
  15. 2025Long-Context State-Space Video World Modelscited by 59
  16. 2025WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Modelcited by 56
  17. 2025Back to the Features: DINO as a Foundation for Video World Modelscited by 56
  18. 2025Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Roboticscited by 47
  19. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  20. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 45
  21. 2025World-in-World: World Models in a Closed-Loop Worldcited by 44
  22. 2025Pre-Trained Video Generative Models as World Simulatorscited by 43
  23. 2025ReSim: Reliable World Simulation for Autonomous Drivingcited by 42
  24. 2026Learning Latent Action World Models In The Wildcited by 42
  25. 2025RLVR-World: Training World Models with Reinforcement Learningcited by 38
  26. 2026Solaris: Building a Multiplayer Video World Model in Minecraftcited by 38
  27. 2025VRAG: Learning World Models for Interactive Video Generationcited by 37
  28. 2025GWM: Towards Scalable Gaussian World Models for Robotic Manipulationcited by 37
  29. 2025DeepVerse: 4D Autoregressive Video Generation as a World Modelcited by 36
  30. 2025PAN: A World Model for General, Actionable, and Long-Horizon World Simulationcited by 36
  31. 2024Diffusion Model Predictive Controlcited by 35
  32. 2024AVID: Adapting Video Diffusion Models to World Modelscited by 35
  33. 2025World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic Manipulationcited by 35
  34. 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modelingcited by 34
  35. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 33
  36. 2025Learning 3D Persistent Embodied World Modelscited by 29
  37. 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environmentscited by 27
  38. 2026Interactive World Simulator for Robot Policy Training and Evaluationcited by 27

50 most cited of 285 All 285 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving Weak World Models Behind Strong Agents in Atari Pong
  2. 2026JEDI: Joint Embedding Diffusion World Model for Online Model-Based Reinforcement Learning
  3. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization
  4. 2025Zero-shot World Models via Search in Memory
  5. 2018World Models
  6. 2024Diffusion Models Are Real-Time Game Engines
  7. 2020Smaller World Models for Reinforcement Learning
  8. 2026Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization