World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models
On the map: World models → Video-generation world models and simulators also fits: Latent predictive world models
Abstract (arXiv)
Video world models must generate physically plausible futures, not just realistic frames. Existing approaches typically use semantic, physical, or predictive information to guide the visual generation stream, leaving a fundamental question: where inside the generator should predictions of scene evolution live? Standard video tokens jointly encode appearance and dynamics, but provide no distinct state for learning and reading predictive dynamics. We answer this question with the Physical Registry: temporally grounded registry slots inside a video DiT that are initialized from an observation-conditioned forecast, co-evolve with video tokens, and receive future-representation supervision, making predictive dynamics an explicit, readable state of the generator. In a controlled comparison of predictive-state interfaces on Physics-IQ Verified, Physical Registry improves vanilla fine-tuning by 9.9/8.7 points on I2V/V2V, the largest gain among the interfaces we compare. Across all Cosmos and WAN backbones, Physical Registry improves Physics-IQ Verified scores. With only 0.45% additional parameters, it enables Cosmos 3 Nano 16B to surpass the published I2V score of Cosmos 3 Super 64B (43.56 vs. 42.7). On WorldModelBench, it also improves physical adherence and the total score for all four evaluated backbones. Because the registry becomes candidate-specific while remaining readable, the Physical Registry Selector (PRS) ranks emerging futures by how far their intermediate registry readouts depart from the shared forecast before full denoising is complete. Across four backbones, this selection improves I2V scores by 5.6-15.9% without completed-video external scoring. Together, these results support a broader principle: dynamics prediction should live as an explicit model state that co-evolves with the video representation. Project page: https://grainsack.github.io/physical-registry/
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 35 of its 43 references that are in the map. First posted on arXiv 12 Oct 2026.
Builds on
35 of 43 references.
Closest ancestors · 7
Same direction, newest first
- 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models14 May
- 2025WorldModelBench: Judging Video Generation Models As World Models28 Feb
- 2024Open-Sora: Democratizing Efficient Video Production for All29 Dec
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024Pandora: Towards General World Model with Natural Language Actions and Video States12 Jun
- 2024Genie: Generative Interactive Environments23 Feb
- 2022Scalable Diffusion Models with Transformers19 Dec
Other references · 28
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2022Video Diffusion Modelscited by 2,305
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2023Vision Transformers Need Registerscited by 871
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
- 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
- 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 163
- 2025Do generative video models understand physical principles?cited by 161
- 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 98
- 2025WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generationcited by 90
16 more
- 2024T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedbackcited by 87
- 2025Intuitive physics understanding emerges from self-supervised pretraining on natural videoscited by 70
- 2025EnerVerse: Envisioning Embodied Future Space for Robotics Manipulationcited by 61
- 2025MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidancecited by 61
- 2025Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signalscited by 53
- 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 52
- 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesiscited by 45
- 2025PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generationcited by 41
- 2026Inference-time Physics Alignment of Video Generative Models with Latent World Modelscited by 37
- 2026Interpreting Physics in Video World Modelscited by 22
- 2025MoAlign: Motion-Centric Representation Alignment for Video Diffusion Modelscited by 16
- 2025Cross-Frame Representation Alignment for Fine-Tuning Video Diffusion Modelscited by 12
- 2026Phantom: Physics-Infused Video Generation via Joint Modeling of Visual and Latent Physical Dynamicscited by 12
- 2026HyperDiT: Hyper-Connected Transformers for High-Fidelity Pixel-Space Diffusioncited by 9
- 2026Physics-IQ Verifiedcited by 9
- 2026PhysVid: Physics Aware Local Conditioning for Generative Video Modelscited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Distilling Physical Priors into Streaming World Models
- 2026Foresight Without Seeing: Latent Futures for World Action Models
- 2026A Mechanistic View on Video Generation as World Models: State and Dynamics
- 2025ProPhy: Progressive Physical Alignment for Dynamic World Simulation
- 2026Nano World Models: A Minimalist Implementation of Future Video Prediction
- 2026PhyWorld: Physics-Faithful World Model for Video Generation
- 2025Simulating the Visual World with Artificial Intelligence: A Roadmap
- 2026A Plausible Future Is Not a Causal Future: Can Video Models Generate How the World Unfolds?