Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
Video--based world models have emerged along two dominant paradigms: video generation and 3D reconstruction. However, existing evaluation benchmarks either focus narrowly on visual fidelity and text--video alignment for generative models, or rely on static 3D reconstruction metrics that fundamentally neglect temporal dynamics. We argue that the future of world modeling lies in 4D generation, which jointly models spatial structure and temporal evolution. In this paradigm, the core capability is interactive response: the ability to faithfully reflect how interaction actions drive state transitions across space and time. Yet no existing benchmark systematically evaluates this critical dimension. To address this gap, we propose Omni--WorldBench, a comprehensive benchmark specifically designed to evaluate the interactive response capabilities of world models in 4D settings. Omni--WorldBench comprises two key components: Omni--WorldSuite, a systematic prompt suite spanning diverse interaction levels and scene types; and Omni--Metrics, an agent-based evaluation framework that quantifies world modeling capabilities by measuring the causal impact of interaction actions on both final outcomes and intermediate state evolution trajectories. We conduct extensive evaluations of 18 representative world models across multiple paradigms. Our analysis reveals critical limitations of current world models in interactive response, providing actionable insights for future research. Omni-WorldBench will be publicly released to foster progress in interactive 4D world modeling.
Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 61 of its 76 references that are in the map. First posted on arXiv 23 Mar 2026.
Builds on
61 of 76 references.
Closest ancestors · 14
Same direction, newest first
- 2026Advancing Open-source World Models28 Jan
- 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
- 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
- 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Prediction25 Sep
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
- 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft11 Apr
- 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness27 Mar
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2025WorldModelBench: Judging Video Generation Models As World Models28 Feb
- 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
- 2024Open-Sora: Democratizing Efficient Video Production for All29 Dec
- 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
2 more
Other references · 47
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2024GPT-4o System Cardcited by 4,467
- 2016Improved Techniques for Training GANscited by 2,938
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2018World Modelscited by 1,444
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2025Qwen-Image Technical Reportcited by 973
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
35 more
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
- 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
- 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
- 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
- 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
- 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
- 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoningcited by 147
- 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
- 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 92
- 2024T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedbackcited by 84
- 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 83
- 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 79
- 2025Large Video Planner Enables Generalizable Robot Controlcited by 67
- 2023Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generationcited by 66
- 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPTcited by 66
- 2025Sekai: A Video Dataset towards World Explorationcited by 64
- 2025A Survey of World Models for Autonomous Drivingcited by 60
- 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 50
- 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 48
- 2026InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulationcited by 48
- 2025RoboScape: Physics-informed Embodied World Modelcited by 38
- 2025Omni-Effects: Unified and Spatially-Controllable Visual Effects Generationcited by 27
- 2026Code2World: A GUI World Model via Renderable Code Generationcited by 26
- 2025FingER: Content Aware Fine-grained Evaluation with Reasoning for AI-Generated Videoscited by 22
- 2025NarrLV: Towards a Comprehensive Narrative-Centric Evaluation for Long Video Generationcited by 21
- 2021Generalizing to the Open World: Deep Visual Odometry with Online Adaptationcited by 19
- 2024A Survey of AI-Generated Video Evaluationcited by 17
- 2025Next Token Is Enough: Realistic Image Quality and Aesthetic Scoring with Multimodal Large Language Modelcited by 11
- 2025ImagerySearch: Adaptive Test-Time Search for Video Generation Beyond Semantic Dependency Constraintscited by 9
- 2024MMGenBench: Fully Automatically Evaluating LMMs from the Text-to-Image Generation Perspectivecited by 4
- 2026Artifact-Aware Evaluation for High-Quality Video Generationcited by 3
- 2026Latent Temporal Discrepancy as Motion Prior: A Loss-Weighting Strategy for Dynamic Fidelity in T2Vcited by 3
Built on by
Citing papers per year: 2026: 17.
17 papers.
Sort:
By area:
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 34
- 2026DreamX-World 1.0: A General-Purpose Interactive World Modelcited by 33
- 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Modelscited by 15
- 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Modelscited by 8
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectivescited by 5
- 2026YoCausal: How Far is Video Generation from World Model? A Causality Perspectivecited by 3
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worldscited by 3
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivitycited by 2
- 2026Apple-π: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligencecited by 1
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstructioncited by 1
- 2026Do Video Generators Track the World Across Segments? A Benchmark and Method for World-State Reasoning in Video Continuationcited by 1
5 more
- 2026Precise Editing and Flexible Referencing for Interactable Worldscited by 1
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Modelcited by 0
- 2026VehDyn: A Driving World Model Benchmark for Vehicle Dynamicscited by 0
- 2026ROWBench: Do Video Models Render What the Program Specifies?cited by 0
- 2026ROWBench: Do Video Models Render What the Program Specifies?1 Oct 2026
- 2026Precise Editing and Flexible Referencing for Interactable Worlds28 Sep 2026
- 2026VehDyn: A Driving World Model Benchmark for Vehicle Dynamics27 Sep 2026
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model16 Sep 2026
- 2026Do Video Generators Track the World Across Segments? A Benchmark and Method for World-State Reasoning in Video Continuation3 Sep 2026
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction28 Aug 2026
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?24 Aug 2026
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worlds17 Aug 2026
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives13 Aug 2026
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity3 Aug 2026
- 2026Apple-π: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligence17 Jul 2026
- 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models30 Jun 2026
5 more
- 2026DreamX-World 1.0: A General-Purpose Interactive World Model15 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Models30 May 2026
- 2026YoCausal: How Far is Video Generation from World Model? A Causality Perspective28 May 2026
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation25 May 2026
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 34
- 2026DreamX-World 1.0: A General-Purpose Interactive World Modelcited by 33
- 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Modelscited by 15
- 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Modelscited by 8
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectivescited by 5
- 2026YoCausal: How Far is Video Generation from World Model? A Causality Perspectivecited by 3
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worldscited by 3
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivitycited by 2
- 2026Apple-π: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligencecited by 1
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstructioncited by 1
- 2026Precise Editing and Flexible Referencing for Interactable Worldscited by 1
12 most cited of 15 All 15 (JSON)
- 2026ROWBench: Do Video Models Render What the Program Specifies?1 Oct 2026
- 2026Precise Editing and Flexible Referencing for Interactable Worlds28 Sep 2026
- 2026VehDyn: A Driving World Model Benchmark for Vehicle Dynamics27 Sep 2026
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction28 Aug 2026
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?24 Aug 2026
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worlds17 Aug 2026
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives13 Aug 2026
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity3 Aug 2026
- 2026Apple-π: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligence17 Jul 2026
- 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models30 Jun 2026
- 2026DreamX-World 1.0: A General-Purpose Interactive World Model15 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
12 newest of 15 All 15 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2026WorldMark: A Unified Benchmark Suite for Interactive Video World Models
- 2026Oneira: From Open-Ended Generation to Open-World Interaction in Video World Models
- 2026HappyWorld-Bench
- 2026iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework
- 2025World-in-World: World Models in a Closed-Loop World
- 2026Trimming the Long-Tail of Visual World Modeling Evaluation
- 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling
- 2025WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World