mlsanityData to 14 Oct 2026

VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation

arXiv 2503.06800 ↗First posted 9 Mar 2025cs.CVcited by 149

Not on the map. Why?

Cited by 39 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large-scale video generative models, capable of creating realistic videos of diverse visual concepts, are strong candidates for general-purpose physical world simulators. However, their adherence to physical commonsense across real-world actions remains unclear (e.g., playing tennis, backflip). Existing benchmarks suffer from limitations such as limited size, lack of human evaluation, sim-to-real gaps, and absence of fine-grained physical rule analysis. To address this, we introduce VideoPhy-2, an action-centric dataset for evaluating physical commonsense in generated videos. We curate 200 diverse actions and detailed prompts for video synthesis from modern generative models. We perform human evaluation that assesses semantic adherence, physical commonsense, and grounding of physical rules in the generated videos. Our findings reveal major shortcomings, with even the best model achieving only 22% joint performance (i.e., high semantic and physical commonsense adherence) on the hard subset of VideoPhy-2. We find that the models particularly struggle with conservation laws like mass and momentum. Finally, we also train VideoPhy-AutoEval, an automatic evaluator for fast, reliable assessment on our dataset. Overall, VideoPhy-2 serves as a rigorous benchmark, exposing critical gaps in video generative models and guiding future research in physically-grounded video generation. The data and code is available at https://videophy2.github.io/.

Key facts. Cited by 149 papers in the map (data through 14 Oct 2026). Builds on 37 of its 62 references that are in the map. First posted on arXiv 9 Mar 2025.

Builds on

37 of 62 references.

References · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  4. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  6. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  7. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  8. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  9. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  10. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  11. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  12. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
25 more
  1. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  2. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  3. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  4. 2024Genie: Generative Interactive Environmentscited by 747
  5. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 670
  6. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 579
  7. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  8. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  9. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  10. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  11. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  12. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  13. 2018A Short Note about Kinetics-600cited by 333
  14. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  15. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
  16. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
  17. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 174
  18. 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 155
  19. 2019PHYRE: A New Benchmark for Physical Reasoningcited by 137
  20. 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 108
  21. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 87
  22. 2025ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Modelscited by 58
  23. 2023VideoCon: Robust Video-Language Alignment via Contrast Captionscited by 30
  24. 2024TALC: Time-Aligned Captions for Multi-Scene Text-to-Video Generationcited by 25
  25. 2022A Survey on Machine Learning Approaches for Modelling Intuitive Physicscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 35; 2026: 114.

149 papers.

Sort:

By area:

  1. 2025World Simulation with Video Foundation Models for Physical AIcited by 200
  2. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  3. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 98
  4. 2025WoW: Towards a World omniscient World model Through Embodied Interactioncited by 67
  5. 2025Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systemscited by 48
  6. 2025PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generationcited by 41
  7. 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 41
  8. 2025VideoScore2: Think before You Score in Generative Video Evaluationcited by 40
  9. 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 39
  10. 2026Inference-time Physics Alignment of Video Generative Models with Latent World Modelscited by 37
  11. 2025"PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Modelscited by 36
  12. 2025PAI-Bench: A Comprehensive Benchmark For Physical AIcited by 34
38 more
  1. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  2. 2025What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewardscited by 25
  3. 2025PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generationcited by 23
  4. 2026AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generationcited by 21
  5. 2025QuantiPhy: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Modelscited by 20
  6. 2025TRAVL: A Recipe for Making Video-Language Models Better Judges of Physics Implausibilitycited by 18
  7. 2026PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Modelscited by 17
  8. 2025Think Before You Diffuse: Infusing Physical Rules into Video Diffusioncited by 16
  9. 2025LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preferencecited by 16
  10. 2025MoAlign: Motion-Centric Representation Alignment for Video Diffusion Modelscited by 16
  11. 2026Self-Refining Video Samplingcited by 15
  12. 2026Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoningcited by 14
  13. 2025Enhancing Physical Plausibility in Video Generation by Reasoning the Implausibilitycited by 13
  14. 2026Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Testcited by 13
  15. 2026Out of Sight, Out of Mind? Evaluating State Evolution in Video World Modelscited by 13
  16. 2026WorldBench: Benchmarking Physical Understanding of World Models by Isolating Physics Conceptscited by 12
  17. 2026Phantom: Physics-Infused Video Generation via Joint Modeling of Visual and Latent Physical Dynamicscited by 12
  18. 2026PhyGround: Benchmarking Physical Reasoning in Generative World Modelscited by 12
  19. 2026NEWTON: Agentic Planning for Physically Grounded Video Generationcited by 12
  20. 2026Current World Models Lack a Persistent State Corecited by 12
  21. 2026MemoBench: Benchmarking World Modeling in Dynamically Changing Environmentscited by 12
  22. 2025ProPhy: Progressive Physical Alignment for Dynamic World Simulationcited by 11
  23. 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Modelscited by 10
  24. 2026RISE-Video: Can Video Generators Decode Implicit World Rules?cited by 10
  25. 2026PhysInOne: Visual Physics Learning and Reasoning in One Suitecited by 9
  26. 2026ConsID-Gen: View-Consistent and Identity-Preserving Image-to-Video Generationcited by 8
  27. 2026World2Act: Latent Action Post-Training from World Model Dynamicscited by 8
  28. 2026Do Joint Audio-Video Generation Models Understand Physics?cited by 8
  29. 2026What-If World: A Causal Benchmark for General World Models in Embodied Scenarioscited by 8
  30. 2026VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine Systemcited by 8
  31. 2025Benchmarking Scientific Understanding and Reasoning for Video Generation using VideoScience-Benchcited by 7
  32. 2025UnityVideo: Unified Multi-Modal Multi-Task Learning for Enhancing World-Aware Video Generationcited by 7
  33. 2026VisPhyWorld: Probing Physical Reasoning via Code-Driven Video Reconstructioncited by 7
  34. 2026Evolution of Video Generative Foundationscited by 7
  35. 2025Planning with Sketch-Guided Verification for Physics-Aware Video Generationcited by 6
  36. 2026PhysVideoGenerator: Towards Physically Aware Video Generation via Latent Physics Guidancecited by 6
  37. 2026PhysVid: Physics Aware Local Conditioning for Generative Video Modelscited by 6
  38. 2026PhyWorld: Physics-Faithful World Model for Video Generationcited by 6

50 most cited of 149 All 149 (JSON)