WorldMark: A Unified Benchmark Suite for Interactive Video World Models
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
Unlike text- or image-driven video generation, an interactive world model is driven by actions: the user acts, and the world responds. Two obstacles stand in the way of fair and comprehensive evaluation. First, models take actions in incompatible formats---captions, camera trajectories, action functions---so no shared protocol has been established. Second, while existing benchmarks have advanced world memory and visual quality, action following is reduced to trajectory or direction error, which collapses a whole path into one number: not how quickly the world reacts to a command switch, nor how cleanly it moves along the commanded axis. WorldMark removes both obstacles. Per-model adapters translate a shared WASD-style vocabulary into each model's native control format, so ten heterogeneous models receive semantically identical instructions across 500 standardized cases spanning styles, viewpoints, and difficulty tiers; a new model costs one adapter. On this common ground we characterize action dynamics through a control-systems lens---direction accuracy, direction purity, response latency, and motion stability, each resolved per axis---alongside suites for world memory and visual quality. Together they expose differences existing protocols cannot see: the fastest responders are often the least stable, a trade-off no single action metric captures; per-axis resolution reveals models that follow translation almost perfectly while barely responding to rotation; the model with the best perceptual and aesthetic quality ranks last in translational direction accuracy and latency; and stylized scenes cost every model global consistency while leaving action dynamics largely intact. We will release all data, evaluation code, and model outputs.
Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 52 of its 64 references that are in the map. First posted on arXiv 23 Apr 2026.
Builds on
52 of 64 references.
Closest ancestors · 18
Same direction, newest first
- 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory10 Apr
- 2026MIND: Benchmarking Memory Consistency and Action Control in World Models8 Feb
- 2026Advancing Open-source World Models28 Jan
- 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
- 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
- 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Models25 Nov
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
- 2025Yume: An Interactive World Generation Model23 Jul
- 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
- 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft11 Apr
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2025WorldModelBench: Judging Video Generation Models As World Models28 Feb
6 more
- 2024Diffusion Models Are Real-Time Game Engines27 Aug
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024Genie: Generative Interactive Environments23 Feb
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
- 2023Learning Interactive Real-World Simulators9 Oct
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environments18 Sep
Other references · 34
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022Video Diffusion Modelscited by 2,280
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2018World Modelscited by 1,444
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
22 more
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
- 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 292
- 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
- 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
- 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 169
- 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 167
- 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 166
- 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
- 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
- 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 92
- 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
- 2022Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracingcited by 78
- 2025Yume-1.5: A Text-Controlled Interactive World Generation Modelcited by 77
- 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
- 2021Aligning Latent and Image Spaces to Connect the Unconnectablecited by 70
- 2026Lyra 2.0: Explorable Generative 3D Worldscited by 36
- 2020EDEN: Multimodal Synthetic Dataset of Enclosed GarDEN Scenescited by 29
- 2025SVBench: Evaluation of Video Generation Models on Social Reasoningcited by 5
Built on by
Citing papers per year: 2026: 24.
24 papers.
Sort:
By area:
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 34
- 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Modelscited by 15
- 2026stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluationcited by 13
- 2026AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Reportcited by 9
- 2026Current World Models Lack a Persistent State Corecited by 8
- 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Modelscited by 8
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectivescited by 5
- 2026A Definition and Roadmap for World Modelscited by 3
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worldscited by 3
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivitycited by 2
- 2026AI for Games in the Foundation Model Eracited by 2
12 more
- 2026Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controlscited by 1
- 2026EO-WM: A Physically Informed World Model for Probabilistic Earth Observation Forecastingcited by 1
- 2026HelloWorld: Enabling Socially Interactive Characters in Video World Modelscited by 1
- 2026WorldRover: A Scalable Synthetic Video Data Engine for World Exploration with Rich Annotationscited by 1
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstructioncited by 1
- 2026Precise Editing and Flexible Referencing for Interactable Worldscited by 1
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Modelcited by 0
- 2026HappyWorld-Benchcited by 0
- 2026CAST: Reconstruction-Coupled Acceleration of Interactive World Modelscited by 0
- 2026MosaiChunk: Compositing Spatio-Temporal Memory for Autoregressive Video Generationcited by 0
- 2026ROWBench: Do Video Models Render What the Program Specifies?cited by 0
- 2026MultiWorldBench: Do Independently Controlled Views Describe One Shared World?cited by 0
- 2026MultiWorldBench: Do Independently Controlled Views Describe One Shared World?8 Oct 2026
- 2026ROWBench: Do Video Models Render What the Program Specifies?1 Oct 2026
- 2026MosaiChunk: Compositing Spatio-Temporal Memory for Autoregressive Video Generation1 Oct 2026
- 2026Precise Editing and Flexible Referencing for Interactable Worlds28 Sep 2026
- 2026CAST: Reconstruction-Coupled Acceleration of Interactive World Models28 Sep 2026
- 2026HappyWorld-Bench21 Sep 2026
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model16 Sep 2026
- 2026AI for Games in the Foundation Model Era15 Sep 2026
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction28 Aug 2026
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worlds17 Aug 2026
- 2026WorldRover: A Scalable Synthetic Video Data Engine for World Exploration with Rich Annotations16 Aug 2026
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives13 Aug 2026
12 more
- 2026HelloWorld: Enabling Socially Interactive Characters in Video World Models5 Aug 2026
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity3 Aug 2026
- 2026AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report20 Jul 2026
- 2026A Definition and Roadmap for World Models7 Jul 2026
- 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models30 Jun 2026
- 2026EO-WM: A Physically Informed World Model for Probabilistic Earth Observation Forecasting25 Jun 2026
- 2026Current World Models Lack a Persistent State Core18 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Models30 May 2026
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation25 May 2026
- 2026stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation20 May 2026
- 2026Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controls19 May 2026
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 34
- 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Modelscited by 15
- 2026stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluationcited by 13
- 2026AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Reportcited by 9
- 2026Current World Models Lack a Persistent State Corecited by 8
- 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Modelscited by 8
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectivescited by 5
- 2026A Definition and Roadmap for World Modelscited by 3
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worldscited by 3
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivitycited by 2
- 2026AI for Games in the Foundation Model Eracited by 2
12 most cited of 19 All 19 (JSON)
- 2026ROWBench: Do Video Models Render What the Program Specifies?1 Oct 2026
- 2026Precise Editing and Flexible Referencing for Interactable Worlds28 Sep 2026
- 2026CAST: Reconstruction-Coupled Acceleration of Interactive World Models28 Sep 2026
- 2026HappyWorld-Bench21 Sep 2026
- 2026AI for Games in the Foundation Model Era15 Sep 2026
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction28 Aug 2026
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worlds17 Aug 2026
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives13 Aug 2026
- 2026HelloWorld: Enabling Socially Interactive Characters in Video World Models5 Aug 2026
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity3 Aug 2026
- 2026AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report20 Jul 2026
- 2026A Definition and Roadmap for World Models7 Jul 2026
12 newest of 19 All 19 (JSON)
- 2026WorldRover: A Scalable Synthetic Video Data Engine for World Exploration with Rich Annotationscited by 1
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Modelcited by 0
- 2026MosaiChunk: Compositing Spatio-Temporal Memory for Autoregressive Video Generationcited by 0
- 2026MultiWorldBench: Do Independently Controlled Views Describe One Shared World?cited by 0
- 2026MultiWorldBench: Do Independently Controlled Views Describe One Shared World?8 Oct 2026
- 2026MosaiChunk: Compositing Spatio-Temporal Memory for Autoregressive Video Generation1 Oct 2026
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model16 Sep 2026
- 2026WorldRover: A Scalable Synthetic Video Data Engine for World Exploration with Rich Annotations16 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models
- 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memory
- 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens
- 2026World Action Models: A Survey
- 2026World Tokens: Enhancing Embodied Policies with Training-Time World Modeling
- 2026WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizon
- 2026Sekai2: From World Exploration to Interactive World Modeling
- 2026World Models as Group Actions