# Interpreting Physics in Video World Models

arXiv: https://arxiv.org/abs/2602.07050 · HTML: https://mlsanity.com/paper/2602.07050/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Latent predictive world models, one of 11 directions in World models.
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Placed in

- World models / [Latent predictive world models](https://mlsanity.com/wm/latent-predictive-world-models/)

## Builds on (most cited first, 22 of 22)

- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [Perception Encoder: The best visual embeddings are not at the output of the network](https://mlsanity.com/paper/2504.13181/) (2025)
- [Revisiting the "Video" in Video-Language Understanding](https://mlsanity.com/paper/2206.01720/) (2022)
- [Do generative video models understand physical principles?](https://mlsanity.com/paper/2501.09038/) (2025)
- [PHYRE: A New Benchmark for Physical Reasoning](https://mlsanity.com/paper/1908.05656/) (2019)
- [Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition](https://mlsanity.com/paper/1912.05534/) (2019)
- [IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning](https://mlsanity.com/paper/1803.07616/) (2018)
- [Intuitive physics understanding emerges from self-supervised pretraining on natural videos](https://mlsanity.com/paper/2502.11831/) (2025)
- [A Comprehensive Survey on World Models for Embodied AI](https://mlsanity.com/paper/2510.16732/) (2025)
- [Understanding Video Transformers via Universal Concept Discovery](https://mlsanity.com/paper/2401.10831/) (2024)
- [Video Time: Properties, Encoders and Evaluation](https://mlsanity.com/paper/1807.06980/) (2018)
- [Phy-Q as a measure for physical reasoning intelligence](https://mlsanity.com/paper/2108.13696/) (2021)
- [Improving the Physics of Video Generation with VJEPA-2 Reward Signal](https://mlsanity.com/paper/2510.21840/) (2025)

## Built on by (most cited first, 22 of 22)

- [RLDX-1 Technical Report](https://mlsanity.com/paper/2605.03269/) (2026)
- [What Makes Video World Model Latents Action-Relevant: Prediction over Reconstruction](https://mlsanity.com/paper/2606.07687/) (2026)
- [QQWorld: Quantile-Quantile Matching for World Model Regularization](https://mlsanity.com/paper/2607.28415/) (2026)
- [Causal Physics Steering in Video World Models via Concept Activation Vectors](https://mlsanity.com/paper/2605.24322/) (2026)
- [RigidBench: Evaluating Rigid-Body Physics in Video Generation Models](https://mlsanity.com/paper/2608.15555/) (2026)
- [Latent Video Prediction Learns Better World Models](https://mlsanity.com/paper/2605.15618/) (2026)
- [What Can Latent World Models Know? Physical Parameter Identifiability in Multimodal Predictive Representations](https://mlsanity.com/paper/2607.27017/) (2026)
- [Principia: Relational Physics Tests for Video Models](https://mlsanity.com/paper/2609.04200/) (2026)
- [Tracing the Arrow of Time: Diagnosing Temporal Information Flow in Video-LLMs](https://mlsanity.com/paper/2605.07568/) (2026)
- [HERA: Historical Evidence Routing Adapter for Physical Prediction in Latent World Models](https://mlsanity.com/paper/2608.05523/) (2026)
- [MultiEcho: An Experimental Science of Learned Worlds](https://mlsanity.com/paper/2609.33347/) (2026)
- [Event-Conditioned Diagnostics of Kinematic, Contact, and Object-Permanence Structure in Passive Object-State World Models](https://mlsanity.com/paper/2606.28455/) (2026)
- [What, Where, and How: Probing Spatiotemporal Representations in Video Foundation Models](https://mlsanity.com/paper/2609.01551/) (2026)
- [One Model, Two Physical Stories: Auditing Misalignment in Multi-Modal World Modeling](https://mlsanity.com/paper/2609.14833/) (2026)
- [A Chosen Future Can Still Be Rewritten: Causal Writability in Video Models](https://mlsanity.com/paper/2609.15980/) (2026)
- [Human-Level Accuracy, Non-Human Strategies: Revealing Model-Human Divergence in Video Physical Reasoning](https://mlsanity.com/paper/2609.22788/) (2026)
- [PhyProbe: Rethinking Physical Consistency Evaluation in Generated Videos](https://mlsanity.com/paper/2609.38377/) (2026)
- [On Parameters of Nonlinear Scalar Dynamics from Video: Invariants, Calibration, and Identifiability](https://mlsanity.com/paper/2609.38877/) (2026)
- [The Planning Limits of Latent World Models](https://mlsanity.com/paper/2609.39235/) (2026)
- [Does Physics Live in the Activations? Localizing Physical Quantities in Video Diffusion Models](https://mlsanity.com/paper/2610.03154/) (2026)
- [Tracking Is Not Permanence: What Video World Models Keep of a Hidden Object](https://mlsanity.com/paper/2610.07355/) (2026)
- [World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models](https://mlsanity.com/paper/2610.16365/) (2026)

## Abstract (arXiv)

A long-standing question in physical reasoning is whether video-based models need to rely on factorized representations of physical variables in order to make physically accurate predictions, or whether they can implicitly represent such variables in a task-specific, distributed manner. While modern video world models achieve strong performance on intuitive physics benchmarks, it remains unclear which of these representational regimes they implement internally. Here, we present the first interpretability study to directly examine physical representations inside large-scale video encoders. Using layerwise probing, subspace geometry, patch-level decoding, and targeted attention ablations, we characterize where physical information becomes accessible and how it is organized within encoder-based video transformers. Across architectures, we identify a sharp intermediate-depth transition -- which we call the Physics Emergence Zone -- at which physical variables become accessible. Physics-related representations peak shortly after this transition and degrade toward the output layers. Decomposing motion into explicit variables, we find that scalar quantities such as speed and acceleration are available from early layers onwards, whereas motion direction becomes accessible only at the Physics Emergence Zone. Notably, we find that direction is encoded through a high-dimensional population structure with circular geometry, requiring coordinated multi-feature intervention to control. These findings suggest that modern video models do not use factorized representations of physical variables like a classical physics engine. Instead, they use a distributed representation that is nonetheless sufficient for making physical predictions.