From Generation to Simulation: How Far Are World Models from Being True Simulators?
On the map: World models → Video-generation world models and simulators
also in 1 more
Abstract (arXiv)
With the rapid progress of diffusion models and large-scale video generation, generative world models are increasingly expected to replace traditional simulators, including physics engines, game engines, and reinforcement-learning environments. Yet the remaining distance from generation to simulation lacks a systematic assessment. We present a capability-based study using an external yardstick: eight capabilities of a traditional simulator, namely asset construction, physics engine, interaction, controllability, stability, state feedback, diversity, and evaluation metrics. We trace three main technical routes--latent dynamics, video generation, and joint-embedding prediction--and map exactly 200 representative works published from 2018 to June 2026 onto these capabilities. Our analysis shows that world models have achieved functional substitution in interaction and controllability for specific scenarios, but remain short of traditional simulators in formal guarantees of physical laws, structured state feedback, and reproducible long-horizon evolution. State feedback is the most neglected cross-route shortcoming: only 6 of 163 implementation papers expose a runtime interface for querying entity states or physical parameters. We identify six research directions: formalized physics, a unified action interface, first-class state feedback, long-horizon stability, downstream-utility evaluation, and cross-route hybridization. Project page: https://github.com/AtongWang/world-model-simulators
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 129 of its 216 references that are in the map. First posted on arXiv 24 Aug 2026.
Builds on
129 of 216 references.
Closest ancestors · 41
Same direction, newest first
- 2026Future Dynamic 3D Reconstruction: Toward 3D World Modeling with Disentangled Ego-Motion16 Jun
- 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memory16 Jun
- 2026DreamX-World 1.0: A General-Purpose Interactive World Model15 Jun
- 2026MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold11 Jun
- 2026Prisma-World: Camera-Controllable Multi-Agent Video World Model8 Jun
- 2026DisCo: World Models with Discrete Camera Motion Control6 Jun
- 2026What Makes Video World Model Latents Action-Relevant: Prediction over Reconstruction5 Jun
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May
- 2026DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memory29 May
- 2026minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models28 May
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation25 May
- 2026WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models24 May
29 more
- 2026Nano World Models: A Minimalist Implementation of Future Video Prediction17 May
- 2026iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework5 May
- 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory10 Apr
- 2026ActionParty: Multi-Subject Action Binding in Generative Video Games2 Apr
- 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms30 Mar
- 2026Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models23 Mar
- 2026Grounding World Simulation Models in a Real-World Metropolis16 Mar
- 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models7 Mar
- 2026Solaris: Building a Multiplayer Video World Model in Minecraft25 Feb
- 2026Fast Autoregressive Video Diffusion and World Models with Temporal Cache Compression and Sparse Attention2 Feb
- 2026Advancing Open-source World Models28 Jan
- 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control8 Jan
- 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos1 Jan
- 2025Memorize-and-Generate: Towards Long-Term Consistency in Real-Time Video Generation21 Dec
- 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
- 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
- 2025WorldPack: Dynamic Frame Compression for Long-context Video World Modeling2 Dec
- 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
- 2025Inferix: A Block-Diffusion based Next-Generation Inference Engine for World Simulation25 Nov
- 2025Simulating the Visual World with Artificial Intelligence: A Roadmap11 Nov
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
- 2025Yume: An Interactive World Generation Model23 Jul
- 2025From 2D to 3D Cognition: A Brief Survey of General World Models25 Jun
- 2025Video World Models with Long-term Spatial Memory5 Jun
- 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft11 Apr
- 2024AVID: Adapting Video Diffusion Models to World Models1 Oct
- 2024Pandora: Towards General World Model with Natural Language Actions and Video States12 Jun
- 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond6 May
- 2021VideoGPT: Video Generation using VQ-VAE and Transformers20 Apr
Other references · 88
- 2018World Modelscited by 1,513
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 367
- 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 300
- 2025Motus: A Unified Latent Action World Modelcited by 251
- 2018Unity: A General Platform for Intelligent Agentscited by 236
- 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 185
- 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 167
- 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
38 more
- 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 137
- 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 135
- 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 106
- 2023ADriver-I: A General World Model for Autonomous Drivingcited by 104
- 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 84
- 2025Yume-1.5: A Text-Controlled Interactive World Generation Modelcited by 84
- 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Drivingcited by 83
- 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
- 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 75
- 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
- 2025WoW: Towards a World omniscient World model Through Embodied Interactioncited by 67
- 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPTcited by 66
- 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AIcited by 62
- 2025A Survey of World Models for Autonomous Drivingcited by 60
- 2024Doe-1: Closed-Loop Autonomous Driving with Large World Modelcited by 51
- 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 51
- 2025World-in-World: World Models in a Closed-Loop Worldcited by 47
- 2026Learning Latent Action World Models In The Wildcited by 47
- 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
- 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
- 2025RoboScape: Physics-informed Embodied World Modelcited by 40
- 2025PAN: A World Model for General, Actionable, and Long-Horizon World Simulationcited by 36
- 2025RoboChallenge: Large-scale Real-robot Evaluation of Embodied Policiescited by 30
- 2025The Role of World Models in Shaping Autonomous Driving: A Comprehensive Surveycited by 29
- 2026ABot-PhysWorld: Interactive World Foundation Model for Robotic Manipulation with Physics Alignmentcited by 29
- 2026GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agentscited by 28
- 2025ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstructioncited by 27
- 2026τ0-WM: A Unified Video-Action World Model for Robotic Manipulationcited by 27
- 2024Playable Game Generationcited by 26
- 2025Learning Real-World Action-Video Dynamics with Heterogeneous Masked Autoregressioncited by 23
- 2025GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generationcited by 23
- 2026RealWonder: Real-Time Physical Action-Conditioned Video Generationcited by 22
- 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Drivingcited by 21
- 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Drivingcited by 20
- 2026BridgeV2W: Bridging Video Generation Models to Embodied World Models via Embodiment Maskscited by 20
- 2025Matrix-Game: Interactive World Foundation Modelcited by 19
- 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 19
- 2026RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulationcited by 19
50 most cited of 88 All 88 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026A Physics-Grounded Benchmark for Multi-Agent Dynamics in World Models
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives
- 2026A Mechanistic View on Video Generation as World Models: State and Dynamics
- 2025Toward Stable World Models: Measuring and Addressing World Instability in Generative Environments
- 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Models
- 2026Physically Native World Models: A Hamiltonian Perspective on Generative World Modeling
- 2026From Pixels to States: Rethinking Interactive World Models as Game Engines
- 2026WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation?