mlsanityData to 14 Oct 2026

WorldModelBench: Judging Video Generation Models As World Models

arXiv 2502.20694 ↗First posted 28 Feb 2025cs.CV cs.AIcited by 124

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Video generation models have rapidly progressed, positioning themselves as video world models capable of supporting decision-making applications like robotics and autonomous driving. However, current benchmarks fail to rigorously evaluate these claims, focusing only on general video quality, ignoring important factors to world models such as physics adherence. To bridge this gap, we propose WorldModelBench, a benchmark designed to evaluate the world modeling capabilities of video generation models in application-driven domains. WorldModelBench offers two key advantages: (1) Against to nuanced world modeling violations: By incorporating instruction-following and physics-adherence dimensions, WorldModelBench detects subtle violations, such as irregular changes in object size that breach the mass conservation law - issues overlooked by prior benchmarks. (2) Aligned with large-scale human preferences: We crowd-source 67K human labels to accurately measure 14 frontier models. Using our high-quality human labels, we further fine-tune an accurate judger to automate the evaluation procedure, achieving 8.6% higher average accuracy in predicting world modeling violations than GPT-4o with 2B parameters. In addition, we demonstrate that training to align human annotations by maximizing the rewards from the judger noticeably improve the world modeling capability. The website is available at https://worldmodelbench-team.github.io.

Key facts. Cited by 124 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 46 of its 60 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

46 of 60 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  2. 2024Pandora: Towards General World Model with Natural Language Actions and Video States12 Jun
  3. 2024Genie: Generative Interactive Environments23 Feb
  4. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  5. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
  6. 2021Learning Transferable Visual Models From Natural Language Supervision26 Feb

Other references · 40

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  8. 2022Video Diffusion Modelscited by 2,305
  9. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  10. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  12. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
28 more
  1. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  2. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  3. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  4. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  5. 2023ModelScope Text-to-Video Technical Reportcited by 658
  6. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
  7. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 579
  8. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
  9. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  10. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 388
  11. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
  12. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  13. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  14. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 331
  15. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 302
  16. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
  17. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 264
  18. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 233
  19. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  20. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 212
  21. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
  22. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 191
  23. 2024Video Diffusion Alignment via Reward Gradientscited by 107
  24. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesiscited by 99
  25. 2024T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedbackcited by 87
  26. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedbackcited by 86
  27. 2024T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Designcited by 54
  28. 2024Towards A Better Metric for Text-to-Video Generationcited by 51

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 25; 2026: 99.

124 papers.

Sort:

By area:

  1. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 156
  2. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
  3. 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Modelscited by 76
  4. 20253D and 4D World Modeling: A Surveycited by 67
  5. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  6. 2025World-in-World: World Models in a Closed-Loop Worldcited by 47
  7. 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 41
  8. 2026RISE: Self-Improving Robot Policy with Compositional World Modelcited by 40
  9. 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 39
  10. 2025VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understandingcited by 32
  11. 2026LLM-as-a-Verifier: A General-Purpose Verification Frameworkcited by 30
  12. 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 28
38 more
  1. 2026WorldMark: A Unified Benchmark Suite for Interactive Video World Modelscited by 26
  2. 2025Embodied AI: From LLMs to World Modelscited by 24
  3. 2026Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Modelscited by 22
  4. 2025Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generationcited by 21
  5. 2025WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planningcited by 20
  6. 2026RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulationcited by 19
  7. 2026GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Trainingcited by 18
  8. 2026Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Modelscited by 18
  9. 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Modelscited by 18
  10. 2025LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preferencecited by 16
  11. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  12. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
  13. 2026iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Frameworkcited by 14
  14. 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memorycited by 14
  15. 2025Position: Interactive Generative Video as Next-Generation Game Enginecited by 13
  16. 2026Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Testcited by 13
  17. 2026Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directionscited by 13
  18. 2026Out of Sight, Out of Mind? Evaluating State Evolution in Video World Modelscited by 13
  19. 2026WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPGcited by 13
  20. 2026PhyGround: Benchmarking Physical Reasoning in Generative World Modelscited by 12
  21. 2026Current World Models Lack a Persistent State Corecited by 12
  22. 2026MemoBench: Benchmarking World Modeling in Dynamically Changing Environmentscited by 12
  23. 2026PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Modelscited by 11
  24. 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Modelscited by 10
  25. 2026PhyCritic: Multimodal Critic Models for Physical AIcited by 10
  26. 2026WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platformcited by 10
  27. 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Modelscited by 10
  28. 2026PhysInOne: Visual Physics Learning and Reasoning in One Suitecited by 9
  29. 2026MiraBench: Evaluating Action-Conditioned Reliability in Robotic World Modelscited by 9
  30. 2026What Makes Video World Model Latents Action-Relevant: Prediction over Reconstructioncited by 9
  31. 2026Kairos: A Regret-Aware Native World-Action Model Stack for Physical AIcited by 9
  32. 2026RoboAlign-R1: Distilled Multimodal Reward Alignment for Robot Video World Modelscited by 8
  33. 2026What-If World: A Causal Benchmark for General World Models in Embodied Scenarioscited by 8
  34. 2025Benchmarking Scientific Understanding and Reasoning for Video Generation using VideoScience-Benchcited by 7
  35. 2026VisPhyWorld: Probing Physical Reasoning via Code-Driven Video Reconstructioncited by 7
  36. 2025Planning with Sketch-Guided Verification for Physics-Aware Video Generationcited by 6
  37. 2026A Mechanistic View on Video Generation as World Models: State and Dynamicscited by 6
  38. 2026Learning Physics-Grounded 4D Dynamics with Neural Gaussian Force Fieldscited by 6

50 most cited of 124 All 124 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024WorldSimBench: Towards Video Generation Models as World Simulators
  2. 2026WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors
  3. 2025Video-Bench: Human-Aligned Video Generation Benchmark
  4. 2025T2VWorldBench: A Benchmark for Evaluating World Knowledge in Text-to-Video Generation
  5. 2026PAWBench: How Far Are We from Probabilistically Aligned World Modeling?
  6. 2025World Consistency Score: A Unified Metric for Video Generation Quality
  7. 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models
  8. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens