mlsanityData to 14 Oct 2026

Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systems

arXiv 2504.02918 ↗First posted 3 Apr 2025cs.CVcited by 48

On the map: World models → Latent predictive world models

Abstract (arXiv)

Recent advances in image and video generation raise hopes that these models possess world modeling capabilities-the ability to generate realistic, physically plausible videos. This could revolutionize applications in robotics, autonomous driving, and scientific simulation. However, before treating these models as world models, we must ask: Do they adhere to physical laws? Current evaluation methods rely on subjective judgments or trajectory matching, limiting their usage for physical reasoning estimation, where many generations could be physically plausible. Thus, we introduce Morpheus, one of the first physics-informed evaluation frameworks for measuring the ability of video generation models to comprehend Newtonian dynamics. Morpheus features 130 real-world videos capturing physical phenomena, guided by conservation laws. Using those as conditioning for video generation, we assess physical plausibility leveraging interpretable metrics evaluated with respect to infallible conservation laws known per physical setting, leveraging advances in physics-informed neural networks and vision-language foundation models. Importantly, Morpheus targets controlled Newtonian rigid-body settings to enable quantitative checks. Our findings reveal that even with advanced prompting and video conditioning, contemporary models struggle to encode physical principles despite generating aesthetically pleasing videos.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 50 of its 61 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

50 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024How Far is Video Generation from World Model: A Physical Law Perspective4 Nov

Other references · 49

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  8. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  9. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  10. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  11. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  12. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
37 more
  1. 2024Depth Anything V2cited by 1,422
  2. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  3. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  4. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  5. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  6. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  7. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
  8. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
  9. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
  10. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
  11. 2023Learning Interactive Real-World Simulatorscited by 458
  12. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  13. 2017Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equationscited by 373
  14. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
  15. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  16. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  17. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  18. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  19. 2019Scaling Autoregressive Video Modelscited by 190
  20. 2024Autoregressive Video Generation without Vector Quantizationcited by 186
  21. 2024Pixtral 12Bcited by 179
  22. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 174
  23. 2023Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuningcited by 170
  24. 2025Do generative video models understand physical principles?cited by 161
  25. 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understandingcited by 160
  26. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 149
  27. 2019PHYRE: A New Benchmark for Physical Reasoningcited by 137
  28. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 106
  29. 2020AI Poincar'e: Machine Learning Conservation Laws from Trajectoriescited by 77
  30. 2023ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Modelscited by 76
  31. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generationcited by 53
  32. 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
  33. 2024TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generationcited by 43
  34. 2024AIGCBench: Comprehensive Evaluation of Image-to-Video Content Generated by AIcited by 27
  35. 2024Mechanistic Neural Networks for Scientific Machine Learningcited by 16
  36. 2024What You See Is What Matters: A Novel Visual and Physics-Based Metric for Evaluating Video Generation Qualitycited by 6
  37. 2025Mechanistic PDE Networks for Discovery of Governing Equationscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 36.

48 papers.

Sort:

By area:

  1. 2025Video models are zero-shot learners and reasonerscited by 220
  2. 2025Are Video Models Ready as Zero-Shot Reasoners? An Empirical Study with the MME-CoF Benchmarkcited by 43
  3. 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 41
  4. 2025"PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Modelscited by 36
  5. 2025What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewardscited by 25
  6. 2025TRAVL: A Recipe for Making Video-Language Models Better Judges of Physics Implausibilitycited by 18
  7. 2025LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preferencecited by 16
  8. 2026Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoningcited by 14
  9. 2026PhyGround: Benchmarking Physical Reasoning in Generative World Modelscited by 12
  10. 2026ACWM-Phys: Investigating Generalized Physical Interaction in Action-Conditioned Video World Modelscited by 10
  11. 2026PhysInOne: Visual Physics Learning and Reasoning in One Suitecited by 9
  12. 2026Video Models Can Reason with Verifiable Rewardscited by 9
36 more
  1. 2026Physics-IQ Verifiedcited by 9
  2. 2026NovaPlan: Zero-Shot Long-Horizon Manipulation via Closed-Loop Video Language Planningcited by 8
  3. 2025Inferring Dynamic Physical Properties from Video Foundation Modelscited by 6
  4. 2025SVBench: Evaluation of Video Generation Models on Social Reasoningcited by 6
  5. 2026Chain of Event-Centric Causal Thought for Physically Plausible Video Generationcited by 6
  6. 2025MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Modelscited by 5
  7. 2025Objects in Generated Videos Are Slower Than They Appear: Models Suffer Sub-Earth Gravity and Don't Know Galileo's Principle...for nowcited by 5
  8. 2025SeeU: Seeing the Unseen World via 4D Dynamics-aware Generationcited by 5
  9. 2026MME-CoF-Pro: Evaluating Reasoning Coherence in Video Generative Models with Text and Visual Hintscited by 5
  10. 2026A Definition and Roadmap for World Modelscited by 5
  11. 2026TRAPSBench: Vision-Language Models Encode but Fail to Express Epistemic Restraintcited by 5
  12. 2026PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignmentcited by 4
  13. 2026CRONOS: Benchmarking Counterfactual Physical Consistency in Video Modelscited by 4
  14. 2026Latent State Design for World Models under Sufficiency Constraintscited by 3
  15. 2026MotionPhysics: Learnable Motion Distillation for Text-Guided Simulationcited by 2
  16. 2026Benchmarking Single-Factor Physical Video-to-Audio Generationcited by 2
  17. 2026Physically Viable World Models: A Case for Query-Conditioned Embodied AIcited by 2
  18. 2026OptiWorld: Optimal Control for Video World Generation under Physical Constraintscited by 2
  19. 2026The Seriality Gap in Video Diffusion Modelscited by 2
  20. 2026NewtPhys: Do Foundation Models Understand Newtonian Physics?cited by 1
  21. 2026Trimming the Long-Tail of Visual World Modeling Evaluationcited by 1
  22. 2026RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysiscited by 1
  23. 2026MultiEcho: An Experimental Science of Learned Worldscited by 1
  24. 2026Foresight at the Event Boundary: Evaluating Physical Prediction in Video World Modelscited by 1
  25. 2026ICAT: Incident-Case-Grounded Adaptive Testing for Physical-Risk Prediction in Embodied World Modelscited by 0
  26. 2026Entity-Centric World Models: Interaction-Aware Masking for Causal Video Predictioncited by 0
  27. 2026WorldString: Actionable World Representationcited by 0
  28. 2026The Perception-Physics Paradox: Probing Scientific Alignment with TC-Benchcited by 0
  29. 2026Einstein World Modelscited by 0
  30. 2026Learning Implicit Causal World Models from Multi-Agent Demonstrationscited by 0
  31. 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Modelcited by 0
  32. 2026On Parameters of Nonlinear Scalar Dynamics from Video: Invariants, Calibration, and Identifiabilitycited by 0
  33. 2026PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Modelscited by 0
  34. 2026Does Physics Live in the Activations? Localizing Physical Quantities in Video Diffusion Modelscited by 0
  35. 2026How Does Geometry Enter Generated Motion?cited by 0
  36. 2026LabInstruct: Benchmarking Situated Instructional Video Generation for Lab Procedurescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026World Models' Last Exam in Physics
  2. 2026WorldBench: Benchmarking Physical Understanding of World Models by Isolating Physics Concepts
  3. 2026Principia: Relational Physics Tests for Video Models
  4. 2026PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Models
  5. 2026Apple-π: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligence
  6. 2025PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation
  7. 2024PhysMotion: Physics-Grounded Dynamics From a Single Image
  8. 2026PhySimCode: A Benchmark and Evaluation Method for Physics Video to Code Generation