mlsanityData to 7 Oct 2026

A Mechanistic View on Video Generation as World Models: State and Dynamics

arXiv 2601.17067 ↗First posted 22 Jan 2026cs.CV cs.AIcited by 6

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Large-scale video generation models have demonstrated emergent physical coherence, positioning them as potential world models. However, a gap remains between contemporary "stateless" video architectures and classic state-centric world model theories. This work bridges this gap by proposing a novel taxonomy centered on two pillars: State Construction and Dynamics Modeling. We categorize state construction into implicit paradigms (context management) and explicit paradigms (latent compression), while dynamics modeling is analyzed through knowledge integration and architectural reformulation. Furthermore, we advocate for a transition in evaluation from visual fidelity to functional benchmarks, testing physical persistence and causal reasoning. We conclude by identifying two critical frontiers: enhancing persistence via data-driven memory and compressed fidelity, and advancing causality through latent factor decoupling and reasoning-prior integration. By addressing these challenges, the field can evolve from generating visually plausible videos to building robust, general-purpose world simulators.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 155 of its 176 references that are in the map. First posted on arXiv 22 Jan 2026.

Builds on

155 of 176 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
  2. 2025EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory1 Oct
  3. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
  4. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
  5. 2025VRAG: Learning World Models for Interactive Video Generation28 May
  6. 2025Long-Context State-Space Video World Models26 May
  7. 2025Vid2World: Crafting Video Diffusion Models to Interactive World Models20 May
  8. 2025MAGI-1: Autoregressive Video Generation at Scale19 May
  9. 2025Video-GPT via Next Clip Diffusion18 May
  10. 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
  11. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  12. 2025WorldModelBench: Judging Video Generation Models As World Models28 Feb
8 more
  1. 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
  2. 2024Open-Sora: Democratizing Efficient Video Production for All29 Dec
  3. 2024Owl-1: Omni World Model for Consistent Long Video Generation12 Dec
  4. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  5. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  6. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  7. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text21 Mar
  8. 2024Genie: Generative Interactive Environments23 Feb

Other references · 135

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  3. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  4. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  5. 2018World Modelscited by 1,444
  6. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  8. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  9. 2023Mastering Diverse Domains through World Modelscited by 1,193
  10. 2022Token Merging: Your ViT But Fastercited by 995
  11. 2020Mastering Atari with Discrete World Modelscited by 990
  12. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
38 more
  1. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  2. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  3. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  4. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  5. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  6. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  7. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  8. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  9. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  10. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  11. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  12. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  13. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  14. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  15. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  16. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  17. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  18. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  19. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  20. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
  21. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  22. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  23. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  24. 2024Titans: Learning to Memorize at Test Timecited by 303
  25. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
  26. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  27. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 247
  28. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  29. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 231
  30. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
  31. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 218
  32. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  33. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 204
  34. 2023Video Understanding with Large Language Models: A Surveycited by 203
  35. 2018Distributed Distributional Deterministic Policy Gradientscited by 195
  36. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  37. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 188
  38. 2024Autoregressive Video Generation without Vector Quantizationcited by 180

50 most cited of 135 All 135 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Simulating the Visual World with Artificial Intelligence: A Roadmap
  2. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms
  3. 2026The Past Frames the Future: Memory for Autoregressive Video Generation
  4. 2024A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approaches
  5. 2026WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors
  6. 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Models
  7. 2026Can Image Models Imagine Time? ImageTime: A Novel Benchmark for Probing Visual World Modeling Through Spatiotemporal Consistency
  8. 2026Persistent Computational State: A Session-Centric Runtime for Generative World Models