# Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoning

arXiv: https://arxiv.org/abs/2603.19607 · HTML: https://mlsanity.com/paper/2603.19607/

## Key facts

- Cited by 13 papers in the map (data through 7 Oct 2026).
- Builds on 45 of its 69 references that are in the map.
- First posted on arXiv 20 Mar 2026.

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Open-Sora: Democratizing Efficient Video Production for All](https://mlsanity.com/paper/2412.20404/) (2024)
- [VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models](https://mlsanity.com/paper/2401.09047/) (2024)
- [Latte: Latent Diffusion Transformer for Video Generation](https://mlsanity.com/paper/2401.03048/) (2024)
- [Open-Sora Plan: Open-Source Large Video Generation Model](https://mlsanity.com/paper/2412.00131/) (2024)
- [The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines](https://mlsanity.com/paper/2005.00343/) (2020)
- [VideoPhy: Evaluating Physical Commonsense for Video Generation](https://mlsanity.com/paper/2406.03520/) (2024)
- [DreamGen: Unlocking Generalization in Robot Learning through Video World Models](https://mlsanity.com/paper/2505.12705/) (2025)
- [Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation](https://mlsanity.com/paper/2410.05363/) (2024)
- [Do generative video models understand physical principles?](https://mlsanity.com/paper/2501.09038/) (2025)
- [HunyuanVideo 1.5 Technical Report](https://mlsanity.com/paper/2511.18870/) (2025)
- [PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding](https://mlsanity.com/paper/2501.16411/) (2025)
- [WorldScore: A Unified Evaluation Benchmark for World Generation](https://mlsanity.com/paper/2504.00983/) (2025)
- [VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation](https://mlsanity.com/paper/2503.06800/) (2025)

## Built on by (most cited first, 13 of 13)

- [World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry](https://mlsanity.com/paper/2604.01985/) (2026)
- [PhyGround: Benchmarking Physical Reasoning in Generative World Models](https://mlsanity.com/paper/2605.10806/) (2026)
- [GS-Agent: Creating 4D Physical Worlds With Generative Simulation](https://mlsanity.com/paper/2607.21522/) (2026)
- [CRONOS: Benchmarking Counterfactual Physical Consistency in Video Models](https://mlsanity.com/paper/2605.23699/) (2026)
- [Physically Viable World Models: A Case for Query-Conditioned Embodied AI](https://mlsanity.com/paper/2605.30542/) (2026)
- [RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysis](https://mlsanity.com/paper/2606.28385/) (2026)
- [Detecting AI-Generated Video: A Vision-Language Dual-View Survey](https://mlsanity.com/paper/2607.10787/) (2026)
- [RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction](https://mlsanity.com/paper/2608.28718/) (2026)
- [VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement](https://mlsanity.com/paper/2609.03153/) (2026)
- [MindTopo: Can Foundation Models Reason in Topological Space?](https://mlsanity.com/paper/2609.11900/) (2026)
- [Foresight at the Event Boundary: Evaluating Physical Prediction in Video World Models](https://mlsanity.com/paper/2609.36531/) (2026)
- [MotionInsight: Diagnosing Object Motion Deficiencies in Generated Videos](https://mlsanity.com/paper/2609.37030/) (2026)
- [PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Models](https://mlsanity.com/paper/2610.01162/) (2026)

## Abstract (arXiv)

Video generation models are increasingly used as world simulators for storytelling, simulation, and embodied AI. As these models advance, a key question arises: do generated videos obey the physical laws of the real world? Existing evaluations largely rely on automated metrics or coarse human judgments such as preferences or rubric-based checks. While useful for assessing perceptual quality, these methods provide limited insight into when and why generated dynamics violate real-world physical constraints. We introduce Physion-Eval, a large-scale benchmark of expert human reasoning for diagnosing physical realism failures in videos generated by five state-of-the-art models across egocentric and exocentric views, containing 10,990 expert reasoning traces spanning 22 fine-grained physical categories. Each generated video is derived from a corresponding real-world reference video depicting a clear physical process, and annotated with temporally localized glitches, structured failure categories, and natural-language explanations of the violated physical behavior. Using this dataset, we reveal a striking limitation of current video generation models: in physics-critical scenarios, 83.3% of exocentric and 93.5% of egocentric generated videos exhibit at least one human-identifiable physical glitch. We hope Physion-Eval will set a new standard for physical realism evaluation and guide the development of physics-grounded video generation. The benchmark is publicly available at https://huggingface.co/datasets/PhysionLabs/Physion-Eval.