mlsanityData to 14 Oct 2026

From Generation to Simulation: How Far Are World Models from Being True Simulators?

arXiv 2608.23070 ↗First posted 24 Aug 2026cs.AI cs.CVcited by 0

On the map: World models → Video-generation world models and simulators

also in 1 more
Abstract (arXiv)

With the rapid progress of diffusion models and large-scale video generation, generative world models are increasingly expected to replace traditional simulators, including physics engines, game engines, and reinforcement-learning environments. Yet the remaining distance from generation to simulation lacks a systematic assessment. We present a capability-based study using an external yardstick: eight capabilities of a traditional simulator, namely asset construction, physics engine, interaction, controllability, stability, state feedback, diversity, and evaluation metrics. We trace three main technical routes--latent dynamics, video generation, and joint-embedding prediction--and map exactly 200 representative works published from 2018 to June 2026 onto these capabilities. Our analysis shows that world models have achieved functional substitution in interaction and controllability for specific scenarios, but remain short of traditional simulators in formal guarantees of physical laws, structured state feedback, and reproducible long-horizon evolution. State feedback is the most neglected cross-route shortcoming: only 6 of 163 implementation papers expose a runtime interface for querying entity states or physical parameters. We identify six research directions: formalized physics, a unified action interface, first-class state feedback, long-horizon stability, downstream-utility evaluation, and cross-route hybridization. Project page: https://github.com/AtongWang/world-model-simulators

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 129 of its 216 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

129 of 216 references.

Closest ancestors · 41

Same direction, newest first

  1. 2026Future Dynamic 3D Reconstruction: Toward 3D World Modeling with Disentangled Ego-Motion16 Jun
  2. 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memory16 Jun
  3. 2026DreamX-World 1.0: A General-Purpose Interactive World Model15 Jun
  4. 2026MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold11 Jun
  5. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Model8 Jun
  6. 2026DisCo: World Models with Discrete Camera Motion Control6 Jun
  7. 2026What Makes Video World Model Latents Action-Relevant: Prediction over Reconstruction5 Jun
  8. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May
  9. 2026DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memory29 May
  10. 2026minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models28 May
  11. 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation25 May
  12. 2026WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models24 May
29 more
  1. 2026Nano World Models: A Minimalist Implementation of Future Video Prediction17 May
  2. 2026iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework5 May
  3. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory10 Apr
  4. 2026ActionParty: Multi-Subject Action Binding in Generative Video Games2 Apr
  5. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms30 Mar
  6. 2026Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models23 Mar
  7. 2026Grounding World Simulation Models in a Real-World Metropolis16 Mar
  8. 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models7 Mar
  9. 2026Solaris: Building a Multiplayer Video World Model in Minecraft25 Feb
  10. 2026Fast Autoregressive Video Diffusion and World Models with Temporal Cache Compression and Sparse Attention2 Feb
  11. 2026Advancing Open-source World Models28 Jan
  12. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control8 Jan
  13. 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos1 Jan
  14. 2025Memorize-and-Generate: Towards Long-Term Consistency in Real-Time Video Generation21 Dec
  15. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
  16. 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
  17. 2025WorldPack: Dynamic Frame Compression for Long-context Video World Modeling2 Dec
  18. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
  19. 2025Inferix: A Block-Diffusion based Next-Generation Inference Engine for World Simulation25 Nov
  20. 2025Simulating the Visual World with Artificial Intelligence: A Roadmap11 Nov
  21. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
  22. 2025Yume: An Interactive World Generation Model23 Jul
  23. 2025From 2D to 3D Cognition: A Brief Survey of General World Models25 Jun
  24. 2025Video World Models with Long-term Spatial Memory5 Jun
  25. 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft11 Apr
  26. 2024AVID: Adapting Video Diffusion Models to World Models1 Oct
  27. 2024Pandora: Towards General World Model with Natural Language Actions and Video States12 Jun
  28. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond6 May
  29. 2021VideoGPT: Video Generation using VQ-VAE and Transformers20 Apr

Other references · 88

  1. 2018World Modelscited by 1,513
  2. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  3. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  4. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  5. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  6. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 367
  7. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 300
  8. 2025Motus: A Unified Latent Action World Modelcited by 251
  9. 2018Unity: A General Platform for Intelligent Agentscited by 236
  10. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 185
  11. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 167
  12. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
38 more
  1. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 137
  2. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 135
  3. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 106
  4. 2023ADriver-I: A General World Model for Autonomous Drivingcited by 104
  5. 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 84
  6. 2025Yume-1.5: A Text-Controlled Interactive World Generation Modelcited by 84
  7. 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Drivingcited by 83
  8. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
  9. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 75
  10. 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
  11. 2025WoW: Towards a World omniscient World model Through Embodied Interactioncited by 67
  12. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPTcited by 66
  13. 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AIcited by 62
  14. 2025A Survey of World Models for Autonomous Drivingcited by 60
  15. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Modelcited by 51
  16. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 51
  17. 2025World-in-World: World Models in a Closed-Loop Worldcited by 47
  18. 2026Learning Latent Action World Models In The Wildcited by 47
  19. 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
  20. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
  21. 2025RoboScape: Physics-informed Embodied World Modelcited by 40
  22. 2025PAN: A World Model for General, Actionable, and Long-Horizon World Simulationcited by 36
  23. 2025RoboChallenge: Large-scale Real-robot Evaluation of Embodied Policiescited by 30
  24. 2025The Role of World Models in Shaping Autonomous Driving: A Comprehensive Surveycited by 29
  25. 2026ABot-PhysWorld: Interactive World Foundation Model for Robotic Manipulation with Physics Alignmentcited by 29
  26. 2026GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agentscited by 28
  27. 2025ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstructioncited by 27
  28. 2026τ0-WM: A Unified Video-Action World Model for Robotic Manipulationcited by 27
  29. 2024Playable Game Generationcited by 26
  30. 2025Learning Real-World Action-Video Dynamics with Heterogeneous Masked Autoregressioncited by 23
  31. 2025GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generationcited by 23
  32. 2026RealWonder: Real-Time Physical Action-Conditioned Video Generationcited by 22
  33. 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Drivingcited by 21
  34. 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Drivingcited by 20
  35. 2026BridgeV2W: Bridging Video Generation Models to Embodied World Models via Embodiment Maskscited by 20
  36. 2025Matrix-Game: Interactive World Foundation Modelcited by 19
  37. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 19
  38. 2026RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulationcited by 19

50 most cited of 88 All 88 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Physics-Grounded Benchmark for Multi-Agent Dynamics in World Models
  2. 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives
  3. 2026A Mechanistic View on Video Generation as World Models: State and Dynamics
  4. 2025Toward Stable World Models: Measuring and Addressing World Instability in Generative Environments
  5. 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Models
  6. 2026Physically Native World Models: A Hamiltonian Perspective on Generative World Modeling
  7. 2026From Pixels to States: Rethinking Interactive World Models as Game Engines
  8. 2026WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation?