mlsanityData to 7 Oct 2026

PISA Experiments: Exploring Physics Post-Training for Video Diffusion Models by Watching Stuff Drop

arXiv 2503.09595 ↗First posted 12 Mar 2025cs.CVcited by 49

On the map: Diffusion models → Video generation

Abstract (arXiv)

Large-scale pre-trained video generation models excel in content creation but are not reliable as physically accurate world simulators out of the box. This work studies the process of post-training these models for accurate world modeling through the lens of the simple, yet fundamental, physics task of modeling object freefall. We show state-of-the-art video generation models struggle with this basic task, despite their visually impressive outputs. To remedy this problem, we find that fine-tuning on a relatively small amount of simulated videos is effective in inducing the dropping behavior in the model, and we can further improve results through a novel reward modeling procedure we introduce. Our study also reveals key limitations of post-training in generalization and distribution modeling. Additionally, we release a benchmark for this task that may serve as a useful diagnostic tool for tracking physical accuracy in large-scale video generative model development.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 26 of its 44 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

26 of 44 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Pyramidal Flow Matching for Efficient Video Generative Modeling8 Oct
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  3. 2024Video Diffusion Alignment via Reward Gradients11 Jul
  4. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors18 Oct

Other references · 22

  1. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  2. 2016Improved Techniques for Training GANscited by 2,938
  3. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2024Depth Anything V2cited by 1,385
  8. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  9. 2023Mastering Diverse Domains through World Modelscited by 1,193
  10. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  11. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  12. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
10 more
  1. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  2. 2023Learning Interactive Real-World Simulatorscited by 440
  3. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  4. 2022Kubric: A scalable dataset generatorcited by 373
  5. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 215
  6. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  7. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 167
  8. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 145
  9. 2024Video as the New Language for Real-World Decision Makingcited by 86
  10. 2016Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networkscited by 46

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 20; 2026: 29.

49 papers.

Sort:

By area:

  1. 2025Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signalscited by 51
  2. 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 40
  3. 2026Solaris: Building a Multiplayer Video World Model in Minecraftcited by 38
  4. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  5. 2025What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewardscited by 23
  6. 2025Epipolar Geometry Improves Video Generation Modelscited by 22
  7. 2025Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generationcited by 20
  8. 2026VGGRPO: Towards World-Consistent Video Generation with 4D Latent Rewardcited by 19
  9. 2025TiViBench: Benchmarking Think-in-Video Reasoning for Video Generative Modelscited by 18
  10. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  11. 2026PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Modelscited by 15
  12. 2026Self-Refining Video Samplingcited by 14
37 more
  1. 2025Enhancing Physical Plausibility in Video Generation by Reasoning the Implausibilitycited by 13
  2. 2026WorldBench: Benchmarking Physical Understanding of World Models by Isolating Physics Conceptscited by 12
  3. 2026Physical Simulator In-the-Loop Video Generationcited by 12
  4. 2025ProPhy: Progressive Physical Alignment for Dynamic World Simulationcited by 11
  5. 2025PhysMaster: Mastering Physical Representation for Video Generation via Reinforcement Learningcited by 10
  6. 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Modelscited by 10
  7. 2026Gamma-World: Generative Multi-Agent World Modeling Beyond Two Playerscited by 10
  8. 2026Goal Force: Teaching Video Models To Accomplish Physics-Conditioned Goalscited by 9
  9. 2026PhysInOne: Visual Physics Learning and Reasoning in One Suitecited by 9
  10. 2026ACWM-Phys: Investigating Generalized Physical Interaction in Action-Conditioned Video World Modelscited by 9
  11. 2026VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine Systemcited by 7
  12. 2025Planning with Sketch-Guided Verification for Physics-Aware Video Generationcited by 6
  13. 2025MoReGen: Multi-Agent Motion-Reasoning Engine for Code-based Text-to-Video Synthesiscited by 5
  14. 2026PhyCo: Learning Controllable Physical Priors for Generative Motioncited by 5
  15. 2025Cloning Deterministic Worlds: The Critical Role of Latent Geometry in Long-Horizon World Modelscited by 3
  16. 2025SeeU: Seeing the Unseen World via 4D Dynamics-aware Generationcited by 3
  17. 2025Over++: Generative Video Compositing for Layer Interaction Effectscited by 3
  18. 2026YoCausal: How Far is Video Generation from World Model? A Causality Perspectivecited by 3
  19. 2026Learning Explicit Physical Parameter Control and Benchmarking for Video Generationcited by 3
  20. 2026GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Modelscited by 3
  21. 2025Learning to Generate Rigid Body Interactions with Video Diffusion Modelscited by 2
  22. 2025VDAWorld: World Modelling via VLM-Directed Abstraction and Simulationcited by 2
  23. 2026PhysEditWorld: A Large-Scale Dataset Toward Physics-Editable World Modelscited by 2
  24. 2025PhysFire-WM: A Physics-Informed World Model for Emulating Fire Spread Dynamicscited by 1
  25. 2026Seeking Physics in Diffusion Noisecited by 1
  26. 2026DiReCT: Disentangled Regularization of Contrastive Trajectories for Physics-Refined Video Generationcited by 1
  27. 2026Physics Question Scene Graph: Fine-grained Evaluation of Physical Plausibility in Text-to-Video Generationcited by 1
  28. 2026NewtPhys: Do Foundation Models Understand Newtonian Physics?cited by 0
  29. 2026PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generationcited by 0
  30. 2026DeforM: Reasoning-Guided Physics-Aware Video Generation via Spatial-Temporal Maskingcited by 0
  31. 2026Diagnosing Under-Development of Irreversible Processes in Video Generationcited by 0
  32. 2026RigidBench: Evaluating Rigid-Body Physics in Video Generation Modelscited by 0
  33. 2026Principia: Relational Physics Tests for Video Modelscited by 0
  34. 2026PhysWeep: Does a Video Generator Realize the Physics You Ask For?cited by 0
  35. 2026MultiEcho: An Experimental Science of Learned Worldscited by 0
  36. 2026HiPhy: Hierarchical Alignment for Physically-Plausible Multi-Principle Video Generationcited by 0
  37. 2026Fluid-Gen-Zero: Grounding Pretrained Video Generators in Physics without Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Show Me: Unifying Instructional Image and Video Generation with Diffusion Models
  2. 2024AVID: Adapting Video Diffusion Models to World Models
  3. 2026Physics-IQ Verified
  4. 2026PhyGround: Benchmarking Physical Reasoning in Generative World Models
  5. 2025Physics-Guided Motion Loss for Video Generation Model
  6. 2026Does Physics Live in the Activations? Localizing Physical Quantities in Video Diffusion Models
  7. 2022Video Diffusion Models
  8. 2025LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preference