# RigidBench: Evaluating Rigid-Body Physics in Video Generation Models

arXiv: https://arxiv.org/abs/2608.15555 · HTML: https://mlsanity.com/paper/2608.15555/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 28 references that are in the map.
- First posted on arXiv 16 Aug 2026.

## Builds on (most cited first, 22 of 22)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [How Far is Video Generation from World Model: A Physical Law Perspective](https://mlsanity.com/paper/2411.02385/) (2024)
- [Video Depth Anything: Consistent Depth Estimation for Super-Long Videos](https://mlsanity.com/paper/2501.12375/) (2025)
- [World Simulation with Video Foundation Models for Physical AI](https://mlsanity.com/paper/2511.00062/) (2025)
- [Do generative video models understand physical principles?](https://mlsanity.com/paper/2501.09038/) (2025)
- [VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation](https://mlsanity.com/paper/2503.06800/) (2025)
- [CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos](https://mlsanity.com/paper/2410.11831/) (2024)
- [WorldModelBench: Judging Video Generation Models As World Models](https://mlsanity.com/paper/2502.20694/) (2025)
- [PISA Experiments: Exploring Physics Post-Training for Video Diffusion Models by Watching Stuff Drop](https://mlsanity.com/paper/2503.09595/) (2025)
- [Interactive World Simulator for Robot Policy Training and Evaluation](https://mlsanity.com/paper/2603.08546/) (2026)
- [World-Gymnast: Training Robots with Reinforcement Learning in a World Model](https://mlsanity.com/paper/2602.02454/) (2026)
- [Interpreting Physics in Video World Models](https://mlsanity.com/paper/2602.07050/) (2026)
- [PhyGround: Benchmarking Physical Reasoning in Generative World Models](https://mlsanity.com/paper/2605.10806/) (2026)
- [Turning Video Models into Generalist Robot Policies](https://mlsanity.com/paper/2605.27817/) (2026)

## Built on by (most cited first, 4 of 4)

- [Does Physics Live in the Activations? Localizing Physical Quantities in Video Diffusion Models](https://mlsanity.com/paper/2610.03154/) (2026)
- [How Does Geometry Enter Generated Motion?](https://mlsanity.com/paper/2610.05135/) (2026)
- [World Models' Last Exam in Physics](https://mlsanity.com/paper/2610.08791/) (2026)
- [PhysCodeGym: Benchmarking Coding Agents for Physically Verifiable 4D Reconstruction](https://mlsanity.com/paper/2610.17288/) (2026)

## Abstract (arXiv)

Video models are increasingly used to predict what happens next in a scene, yet the metrics commonly used to compare their outputs say little about whether the predicted objects move correctly. Motion, geometry, identity, background stability, and visual similarity can fail independently, but whole-frame scores often mix these errors together. We introduce RigidBench, a simulator-grounded benchmark that compares a generated continuation with a reference rollout from the same initial frame and motion description. Its five rigid-body tasks vary objects, materials, viewpoints, and indoor and outdoor scenes, with per-frame masks, depth, 6-DoF trajectories, and contacts available for scoring. We evaluate eight models on the same 100 examples with ten measurements that keep these aspects separate. The resulting rankings depend strongly on what is measured: no model leads on all ten, and across model means, higher SSIM accompanies larger 3D trajectory error (r = 0.89). RigidBench also includes 5,000 training videos with exact simulator state, which we use to fine-tune and analyze Wan 2.2 TI2V-5B. Full fine-tuning reduces 3D trajectory error by about 20% with almost no change in SSIM, while teacher-forced probes and targeted interventions show that object position is represented throughout Wan's diffusion transformer and used by its denoising computation.