A Mechanistic View on Video Generation as World Models: State and Dynamics
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
Large-scale video generation models have demonstrated emergent physical coherence, positioning them as potential world models. However, a gap remains between contemporary "stateless" video architectures and classic state-centric world model theories. This work bridges this gap by proposing a novel taxonomy centered on two pillars: State Construction and Dynamics Modeling. We categorize state construction into implicit paradigms (context management) and explicit paradigms (latent compression), while dynamics modeling is analyzed through knowledge integration and architectural reformulation. Furthermore, we advocate for a transition in evaluation from visual fidelity to functional benchmarks, testing physical persistence and causal reasoning. We conclude by identifying two critical frontiers: enhancing persistence via data-driven memory and compressed fidelity, and advancing causality through latent factor decoupling and reasoning-prior integration. By addressing these challenges, the field can evolve from generating visually plausible videos to building robust, general-purpose world simulators.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 155 of its 176 references that are in the map. First posted on arXiv 22 Jan 2026.
Builds on
155 of 176 references.
Closest ancestors · 20
Same direction, newest first
- 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
- 2025EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory1 Oct
- 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
- 2025VRAG: Learning World Models for Interactive Video Generation28 May
- 2025Long-Context State-Space Video World Models26 May
- 2025Vid2World: Crafting Video Diffusion Models to Interactive World Models20 May
- 2025MAGI-1: Autoregressive Video Generation at Scale19 May
- 2025Video-GPT via Next Clip Diffusion18 May
- 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2025WorldModelBench: Judging Video Generation Models As World Models28 Feb
8 more
- 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
- 2024Open-Sora: Democratizing Efficient Video Production for All29 Dec
- 2024Owl-1: Omni World Model for Consistent Long Video Generation12 Dec
- 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
- 2024Diffusion Models Are Real-Time Game Engines27 Aug
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text21 Mar
- 2024Genie: Generative Interactive Environments23 Feb
Other references · 135
- 2023A Survey of Large Language Modelscited by 2,537
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2018World Modelscited by 1,444
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2022Token Merging: Your ViT But Fastercited by 995
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
38 more
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
- 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
- 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
- 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
- 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
- 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
- 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
- 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
- 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
- 2023Learning to Compress Prompts with Gist Tokenscited by 379
- 2022Temporal Difference Learning for Model Predictive Controlcited by 375
- 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
- 2023Photorealistic Video Generation with Diffusion Modelscited by 304
- 2024Titans: Learning to Memorize at Test Timecited by 303
- 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
- 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
- 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 247
- 2025LongLive: Real-time Interactive Long Video Generationcited by 237
- 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 231
- 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
- 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 218
- 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
- 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 204
- 2023Video Understanding with Large Language Models: A Surveycited by 203
- 2018Distributed Distributional Deterministic Policy Gradientscited by 195
- 2023Knowledge Editing for Large Language Models: A Surveycited by 189
- 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 188
- 2024Autoregressive Video Generation without Vector Quantizationcited by 180
50 most cited of 135 All 135 (JSON)
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPGcited by 13
- 2026The Pulse of Motion: Measuring Physical Frame Rate from Visual Dynamicscited by 7
- 2026Marionette: Predicting World States, Rendering Geometry, Painting Appearancecited by 5
- 2026From Pixels to States: Rethinking Interactive World Models as Game Enginescited by 4
- 2026Looped World Modelscited by 2
- 2026IntactWorld: Joint World Modeling with Intact Featurescited by 0
- 2026IntactWorld: Joint World Modeling with Intact Features8 Oct 2026
- 2026Marionette: Predicting World States, Rendering Geometry, Painting Appearance14 Aug 2026
- 2026From Pixels to States: Rethinking Interactive World Models as Game Engines15 Jul 2026
- 2026Looped World Models16 Jun 2026
- 2026WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG24 Mar 2026
- 2026The Pulse of Motion: Measuring Physical Frame Rate from Visual Dynamics15 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2025Simulating the Visual World with Artificial Intelligence: A Roadmap
- 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms
- 2026The Past Frames the Future: Memory for Autoregressive Video Generation
- 2024A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approaches
- 2026WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors
- 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Models
- 2026Can Image Models Imagine Time? ImageTime: A Novel Benchmark for Probing Visual World Modeling Through Spatiotemporal Consistency
- 2026Persistent Computational State: A Session-Centric Runtime for Generative World Models