# Does Physics Live in the Activations? Localizing Physical Quantities in Video Diffusion Models

arXiv: https://arxiv.org/abs/2610.03154 · HTML: https://mlsanity.com/paper/2610.03154/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 24 of its 28 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)
- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 24 of 24)

- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [How Far is Video Generation from World Model: A Physical Law Perspective](https://mlsanity.com/paper/2411.02385/) (2024)
- [Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation](https://mlsanity.com/paper/2410.05363/) (2024)
- [Do generative video models understand physical principles?](https://mlsanity.com/paper/2501.09038/) (2025)
- [Video as the New Language for Real-World Decision Making](https://mlsanity.com/paper/2402.17139/) (2024)
- [IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning](https://mlsanity.com/paper/1803.07616/) (2018)
- [Video Diffusion Models are Training-free Motion Interpreter and Controller](https://mlsanity.com/paper/2405.14864/) (2024)
- [Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systems](https://mlsanity.com/paper/2504.02918/) (2025)
- [T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generation](https://mlsanity.com/paper/2505.00337/) (2025)
- ["PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Models](https://mlsanity.com/paper/2507.13428/) (2025)
- [Interpreting Physics in Video World Models](https://mlsanity.com/paper/2602.07050/) (2026)
- [The Invisible Hand of Physics: When Video Diffusion Models Know More Than They Show](https://mlsanity.com/paper/2606.05328/) (2026)
- [PhyEduVideo: A Benchmark for Evaluating Text-to-Video Models for Physics Education](https://mlsanity.com/paper/2601.00943/) (2026)
- [RigidBench: Evaluating Rigid-Body Physics in Video Generation Models](https://mlsanity.com/paper/2608.15555/) (2026)
- [Pixels to Principles: Probing Intuitive Physics Understanding in Multimodal Language Models](https://mlsanity.com/paper/2507.16572/) (2025)
- [Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Control](https://mlsanity.com/paper/2606.04775/) (2026)
- [How Do Video Foundation Models Encode Intuitive Physics? Probing Across Pretraining Paradigms](https://mlsanity.com/paper/2606.09646/) (2026)
- [CaliBench: Are the Stochastic Dynamics of Video World Models Physically Calibrated?](https://mlsanity.com/paper/2608.16829/) (2026)
- [Steering Video Diffusion Transformers with Massive Activations](https://mlsanity.com/paper/2603.17825/) (2026)
- [TunerDiT: Training-free Progressive Steering of Diffusion Transformer for Multi-Event Video Generation](https://mlsanity.com/paper/2605.31590/) (2026)
- [Why Do Video Diffusion Models Violate Physics? Unveiling the Flaws in Attention Mechanisms](https://mlsanity.com/paper/2609.23658/) (2026)
- [TokenDial: Continuous Attribute Control for Text-to-Video Generation in Visual Dial Space](https://mlsanity.com/paper/2603.27520/) (2026)
- [Pulling The REINS: Training-Free Safety Alignment of Video Diffusion Models via Representation Steering](https://mlsanity.com/paper/2606.17257/) (2026)

## Abstract (arXiv)

Video generation models produce strikingly realistic sequences and are increasingly proposed as world models, yet recent benchmarks reveal pronounced deficits in their physical reasoning. This raises the question of whether these models internalize physical principles or merely reproduce familiar motion patterns. We address this by probing internal representations of video Diffusion Transformers (DiTs) for simulator-derived ground-truth physical quantities spanning kinematic motion and rigid-body dynamics under gravity and contact. We find that these quantities are linearly decodable with high accuracy early in the denoising process, substantially outperforming a baseline decoded directly from the model's own noised latents, indicating that the relevant physical information is actively constructed during denoising rather than already present in the input. Additionally, we show that activations at on-object tokens carry the relevant physical information and that quantities defined over multiple frames are readable from single latent frames. Hence, information is sharply localized within the token sequence and is computed globally but stored locally. The probes further show partial extrapolation, transferring to scene variations and object configurations outside their training regime, so what they read is not simply a correlate of the scenes they were fit on. When fitted directly in the full-resolution activation space, the probing directions can serve as steering vectors to change the model's output.