mlsanityData to 7 Oct 2026

WorldMark: A Unified Benchmark Suite for Interactive Video World Models

arXiv 2604.21686 ↗First posted 23 Apr 2026cs.CVcited by 24

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Unlike text- or image-driven video generation, an interactive world model is driven by actions: the user acts, and the world responds. Two obstacles stand in the way of fair and comprehensive evaluation. First, models take actions in incompatible formats---captions, camera trajectories, action functions---so no shared protocol has been established. Second, while existing benchmarks have advanced world memory and visual quality, action following is reduced to trajectory or direction error, which collapses a whole path into one number: not how quickly the world reacts to a command switch, nor how cleanly it moves along the commanded axis. WorldMark removes both obstacles. Per-model adapters translate a shared WASD-style vocabulary into each model's native control format, so ten heterogeneous models receive semantically identical instructions across 500 standardized cases spanning styles, viewpoints, and difficulty tiers; a new model costs one adapter. On this common ground we characterize action dynamics through a control-systems lens---direction accuracy, direction purity, response latency, and motion stability, each resolved per axis---alongside suites for world memory and visual quality. Together they expose differences existing protocols cannot see: the fastest responders are often the least stable, a trade-off no single action metric captures; per-axis resolution reveals models that follow translation almost perfectly while barely responding to rotation; the model with the best perceptual and aesthetic quality ranks last in translational direction accuracy and latency; and stylized scenes cost every model global consistency while leaving action dynamics largely intact. We will release all data, evaluation code, and model outputs.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 52 of its 64 references that are in the map. First posted on arXiv 23 Apr 2026.

Builds on

52 of 64 references.

Closest ancestors · 18

Same direction, newest first

  1. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory10 Apr
  2. 2026MIND: Benchmarking Memory Consistency and Action Control in World Models8 Feb
  3. 2026Advancing Open-source World Models28 Jan
  4. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
  5. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
  6. 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Models25 Nov
  7. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
  8. 2025Yume: An Interactive World Generation Model23 Jul
  9. 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
  10. 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft11 Apr
  11. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  12. 2025WorldModelBench: Judging Video Generation Models As World Models28 Feb
6 more
  1. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  3. 2024Genie: Generative Interactive Environments23 Feb
  4. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  5. 2023Learning Interactive Real-World Simulators9 Oct
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environments18 Sep

Other references · 34

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2022Video Diffusion Modelscited by 2,280
  5. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  6. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  7. 2018World Modelscited by 1,444
  8. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  9. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  10. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  11. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  12. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
22 more
  1. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  2. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
  3. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  4. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  5. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  6. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 292
  7. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  8. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
  9. 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 169
  10. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 167
  11. 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 166
  12. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  13. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  14. 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 92
  15. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
  16. 2022Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracingcited by 78
  17. 2025Yume-1.5: A Text-Controlled Interactive World Generation Modelcited by 77
  18. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  19. 2021Aligning Latent and Image Spaces to Connect the Unconnectablecited by 70
  20. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 36
  21. 2020EDEN: Multimodal Synthetic Dataset of Enclosed GarDEN Scenescited by 29
  22. 2025SVBench: Evaluation of Video Generation Models on Social Reasoningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 24.

24 papers.

Sort:

By area:

  1. 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 34
  2. 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Modelscited by 15
  3. 2026stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluationcited by 13
  4. 2026AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Reportcited by 9
  5. 2026Current World Models Lack a Persistent State Corecited by 8
  6. 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Modelscited by 8
  7. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
  8. 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectivescited by 5
  9. 2026A Definition and Roadmap for World Modelscited by 3
  10. 2026HarnessEval-W: Agentifying the Evaluation of Visual Worldscited by 3
  11. 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivitycited by 2
  12. 2026AI for Games in the Foundation Model Eracited by 2
12 more
  1. 2026Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controlscited by 1
  2. 2026EO-WM: A Physically Informed World Model for Probabilistic Earth Observation Forecastingcited by 1
  3. 2026HelloWorld: Enabling Socially Interactive Characters in Video World Modelscited by 1
  4. 2026WorldRover: A Scalable Synthetic Video Data Engine for World Exploration with Rich Annotationscited by 1
  5. 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstructioncited by 1
  6. 2026Precise Editing and Flexible Referencing for Interactable Worldscited by 1
  7. 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Modelcited by 0
  8. 2026HappyWorld-Benchcited by 0
  9. 2026CAST: Reconstruction-Coupled Acceleration of Interactive World Modelscited by 0
  10. 2026MosaiChunk: Compositing Spatio-Temporal Memory for Autoregressive Video Generationcited by 0
  11. 2026ROWBench: Do Video Models Render What the Program Specifies?cited by 0
  12. 2026MultiWorldBench: Do Independently Controlled Views Describe One Shared World?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models
  2. 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memory
  3. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens
  4. 2026World Action Models: A Survey
  5. 2026World Tokens: Enhancing Embodied Policies with Training-Time World Modeling
  6. 2026WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizon
  7. 2026Sekai2: From World Exploration to Interactive World Modeling
  8. 2026World Models as Group Actions