# Inference-time Physics Alignment of Video Generative Models with Latent World Models

arXiv: https://arxiv.org/abs/2601.10553 · HTML: https://mlsanity.com/paper/2601.10553/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Latent predictive world models, one of 11 directions in World models.
- Builds on 59 of its 75 references that are in the map.
- First posted on arXiv 15 Jan 2026.

## Placed in

- World models / [Latent predictive world models](https://mlsanity.com/wm/latent-predictive-world-models/)
- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 30 of 59)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Movie Gen: A Cast of Media Foundation Models](https://mlsanity.com/paper/2410.13720/) (2024)
- [GAIA-1: A Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2309.17080/) (2023)
- [VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models](https://mlsanity.com/paper/2401.09047/) (2024)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023)
- [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023)
- [Evaluating Text-to-Visual Generation with Image-to-Text Generation](https://mlsanity.com/paper/2404.01291/) (2024)
- [Learning Interactive Real-World Simulators](https://mlsanity.com/paper/2310.06114/) (2023)
- [Universal Guidance for Diffusion Models](https://mlsanity.com/paper/2302.07121/) (2023)
- [LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models](https://mlsanity.com/paper/2309.15103/) (2023)
- [MAGI-1: Autoregressive Video Generation at Scale](https://mlsanity.com/paper/2505.13211/) (2025)
- [Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps](https://mlsanity.com/paper/2501.09732/) (2025)
- [Practical and Asymptotically Exact Conditional Sampling in Diffusion Models](https://mlsanity.com/paper/2306.17775/) (2023)
- [VideoPhy: Evaluating Physical Commonsense for Video Generation](https://mlsanity.com/paper/2406.03520/) (2024)
- [A General Framework for Inference-time Scaling and Steering of Diffusion Models](https://mlsanity.com/paper/2501.06848/) (2025)
- [How Far is Video Generation from World Model: A Physical Law Perspective](https://mlsanity.com/paper/2411.02385/) (2024)
- [DreamGen: Unlocking Generalization in Robot Learning through Video World Models](https://mlsanity.com/paper/2505.12705/) (2025)

## Built on by (most cited first, 30 of 37)

- [Cosmos 3: Omnimodal World Models for Physical AI](https://mlsanity.com/paper/2606.02800/) (2026)
- [Solaris: Building a Multiplayer Video World Model in Minecraft](https://mlsanity.com/paper/2602.22208/) (2026)
- [Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players](https://mlsanity.com/paper/2605.28816/) (2026)
- [Prisma-World: Camera-Controllable Multi-Agent Video World Model](https://mlsanity.com/paper/2606.09507/) (2026)
- [Physics-IQ Verified](https://mlsanity.com/paper/2606.18943/) (2026)
- [PhyWorld: Physics-Faithful World Model for Video Generation](https://mlsanity.com/paper/2605.19242/) (2026)
- [Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation](https://mlsanity.com/paper/2605.28230/) (2026)
- [Human Cognition in Machines: A Unified Perspective of World Models](https://mlsanity.com/paper/2604.16592/) (2026)
- [PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignment](https://mlsanity.com/paper/2603.13770/) (2026)
- [Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation](https://mlsanity.com/paper/2605.13111/) (2026)
- [PhysOmni: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction](https://mlsanity.com/paper/2605.20290/) (2026)
- [MMPhysVideo: Physically Plausible Video Generation Through Joint RGB-Perception Modeling](https://mlsanity.com/paper/2604.02817/) (2026)
- [Physics in 2-Steps: Locking Motion Priors Before Visual Refinement Erases Them](https://mlsanity.com/paper/2606.06361/) (2026)
- [GEOPHYS: The Geometry of Physical Plausibility](https://mlsanity.com/paper/2606.20707/) (2026)
- [CaliBench: Are the Stochastic Dynamics of Video World Models Physically Calibrated?](https://mlsanity.com/paper/2608.16829/) (2026)
- [Visko Orbis 1.0: A Live Model for Real-Time Interactive Long Video Generation](https://mlsanity.com/paper/2607.26694/) (2026)
- [Off-Manifold Refinement: Guiding Video Generators with a Frozen World Model](https://mlsanity.com/paper/2608.29904/) (2026)
- [Why Do Video Diffusion Models Violate Physics? Unveiling the Flaws in Attention Mechanisms](https://mlsanity.com/paper/2609.23658/) (2026)
- [Seeking Physics in Diffusion Noise](https://mlsanity.com/paper/2603.14294/) (2026)
- [MotiMotion: Motion-Controlled Video Generation with Visual Reasoning](https://mlsanity.com/paper/2605.22818/) (2026)
- [StressDream: Steering Video World Models for Robust Policy Evaluation and Improvement](https://mlsanity.com/paper/2606.00267/) (2026)
- [Physics-Informed Video Generation via Mixture-of-Experts Latent Alignment](https://mlsanity.com/paper/2606.04737/) (2026)
- [PAVXploreRL: Physical-Action-Visual World Model Reinforcement Learning with Action Exploration](https://mlsanity.com/paper/2607.16602/) (2026)
- [SCOPE: Score-Isolated Agentic Optimization for Video World Models](https://mlsanity.com/paper/2608.15043/) (2026)
- [Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioning](https://mlsanity.com/paper/2609.00656/) (2026)
- [Sampling headroom is not selection gain: a compute-value audit of test-time scaling for video world models](https://mlsanity.com/paper/2609.13257/) (2026)
- [A Chosen Future Can Still Be Rewritten: Causal Writability in Video Models](https://mlsanity.com/paper/2609.15980/) (2026)
- [Beyond Visual Quality: A Study of Test-Time Planning with World Action Models](https://mlsanity.com/paper/2609.24745/) (2026)
- [Reimagine Video Dynamics](https://mlsanity.com/paper/2609.36496/) (2026)
- [PhyProbe: Rethinking Physical Consistency Evaluation in Generated Videos](https://mlsanity.com/paper/2609.38377/) (2026)

## Abstract (arXiv)

State-of-the-art video generative models produce promising visual content yet often violate basic physics principles, limiting their utility. While some attribute this deficiency to insufficient physics understanding from pre-training, we find that the shortfall in physics plausibility also stems from suboptimal inference strategies. We therefore introduce WMReward and treat improving physics plausibility of video generation as an inference-time alignment problem. In particular, we leverage the strong physics prior of a latent world model (here, VJEPA-2) as a reward to search and steer multiple candidate denoising trajectories, enabling scaling test-time compute for better generation performance. Empirically, our approach substantially improves physics plausibility across image-conditioned, multiframe-conditioned, and text-conditioned generation settings, with validation from human preference study. Notably, in the ICCV 2025 Perception Test PhysicsIQ Challenge, we achieve a final score of 62.64%, winning first place and outperforming the previous state of the art by 7.42%. Our work demonstrates the viability of using latent world models to improve physics plausibility of video generation, beyond this specific instantiation or parameterization.