mlsanityData to 7 Oct 2026

Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms

arXiv 2603.28489 ↗First posted 30 Mar 2026eess.IV cs.CVcited by 5

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

The rapid evolution of video generation has enabled models to simulate complex physical dynamics and long-horizon causalities, positioning them as potential world simulators. However, a critical gap still remains between the theoretical capacity for world simulation and the heavy computational costs of spatiotemporal modeling. To address this, we comprehensively and systematically review video generation frameworks and techniques that consider efficiency as a crucial requirement for practical world modeling. We introduce a novel taxonomy in three dimensions: efficient modeling paradigms, efficient network architectures, and efficient inference algorithms. We further show that bridging this efficiency gap directly empowers interactive applications such as autonomous driving, embodied AI, and game simulation. Finally, we identify emerging research frontiers in efficient video-based world modeling, arguing that efficiency is a fundamental prerequisite for evolving video generators into general-purpose, real-time, and robust world simulators. A curated GitHub repository of the reviewed literature can be found at https://github.com/Isaachhh/Efficient-VWM-Survey.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 238 of its 254 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

238 of 254 references.

Closest ancestors · 27

Same direction, newest first

  1. 2026Advancing Open-source World Models28 Jan
  2. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
  3. 2025Astra: General Interactive World Model with Autoregressive Denoising9 Dec
  4. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
  5. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraft3 Oct
  6. 2025EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory1 Oct
  7. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
  8. 2025HERO: Hierarchical Extrapolation and Refresh for Efficient World Models25 Aug
  9. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
  10. 2025Video World Models with Long-term Spatial Memory5 Jun
  11. 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft11 Apr
  12. 2025Exploration-Driven Generative Interactive Environments3 Apr
15 more
  1. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  2. 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
  3. 2024Open-Sora: Democratizing Efficient Video Production for All29 Dec
  4. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  5. 2024WorldSimBench: Towards Video Generation Models as World Simulators23 Oct
  6. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  7. 2024Genie: Generative Interactive Environments23 Feb
  8. 2023Learning Interactive Real-World Simulators9 Oct
  9. 2022Scalable Diffusion Models with Transformers19 Dec
  10. 2022MAGVIT: Masked Generative Video Transformer10 Dec
  11. 2022Flow Matching for Generative Modeling6 Oct
  12. 2021VideoGPT: Video Generation using VQ-VAE and Transformers20 Apr
  13. 2021Learning Transferable Visual Models From Natural Language Supervision26 Feb
  14. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  15. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 211

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  11. 2022Video Diffusion Modelscited by 2,280
  12. 2023Consistency Modelscited by 1,778
38 more
  1. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  2. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2018World Modelscited by 1,444
  5. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  6. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  7. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  8. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  9. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  10. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  11. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  12. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  13. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  14. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  15. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  16. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  17. 2026World Action Models are Zero-shot Policiescited by 411
  18. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  19. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  20. 2024One Step Diffusion via Shortcut Modelscited by 373
  21. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
  22. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  23. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  24. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
  25. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 311
  26. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  27. 2026Causal World Modeling for Robot Controlcited by 289
  28. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  29. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 258
  30. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 247
  31. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 245
  32. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  33. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  34. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
  35. 2023Token Merging for Fast Stable Diffusioncited by 220
  36. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 204
  37. 2016Temporal Generative Adversarial Nets with Singular Value Clippingcited by 198
  38. 2025Unified Video Action Modelcited by 195

50 most cited of 211 All 211 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Mechanistic View on Video Generation as World Models: State and Dynamics
  2. 2025Simulating the Visual World with Artificial Intelligence: A Roadmap
  3. 2024Exploring the Interplay Between Video Generation and World Models in Autonomous Driving: A Survey
  4. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond
  5. 2026Autonomous Video Generation with Counterfactual Controllability for Self-Evolving World Models
  6. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generation
  7. 2026PAWBench: How Far Are We from Probabilistically Aligned World Modeling?
  8. 2026Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directions