mlsanityData to 14 Oct 2026

Inference-time Physics Alignment of Video Generative Models with Latent World Models

arXiv 2601.10553 ↗First posted 15 Jan 2026cs.CVcited by 37

On the map: World models → Latent predictive world models

also in 1 more
Abstract (arXiv)

State-of-the-art video generative models produce promising visual content yet often violate basic physics principles, limiting their utility. While some attribute this deficiency to insufficient physics understanding from pre-training, we find that the shortfall in physics plausibility also stems from suboptimal inference strategies. We therefore introduce WMReward and treat improving physics plausibility of video generation as an inference-time alignment problem. In particular, we leverage the strong physics prior of a latent world model (here, VJEPA-2) as a reward to search and steer multiple candidate denoising trajectories, enabling scaling test-time compute for better generation performance. Empirically, our approach substantially improves physics plausibility across image-conditioned, multiframe-conditioned, and text-conditioned generation settings, with validation from human preference study. Notably, in the ICCV 2025 Perception Test PhysicsIQ Challenge, we achieve a final score of 62.64%, winning first place and outperforming the previous state of the art by 7.42%. Our work demonstrates the viability of using latent world models to improve physics plausibility of video generation, beyond this specific instantiation or parameterization.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 59 of its 75 references that are in the map. First posted on arXiv 15 Jan 2026.

Builds on

59 of 75 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024How Far is Video Generation from World Model: A Physical Law Perspective4 Nov
  2. 2024Revisiting Feature Prediction for Learning Visual Representations from Video15 Feb

Other references · 57

  1. 2020Denoising Diffusion Implicit Modelscited by 9,781
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2022Flow Matching for Generative Modelingcited by 5,702
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  7. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  9. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  10. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  11. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  12. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
38 more
  1. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  2. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  3. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  4. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 579
  5. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
  6. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  7. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  8. 2023Learning Interactive Real-World Simulatorscited by 458
  9. 2023Universal Guidance for Diffusion Modelscited by 437
  10. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  11. 2025MAGI-1: Autoregressive Video Generation at Scalecited by 276
  12. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 234
  13. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Modelscited by 232
  14. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  15. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 224
  16. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 179
  17. 2025Do generative video models understand physical principles?cited by 161
  18. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 153
  19. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 149
  20. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 136
  21. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 117
  22. 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 113
  23. 2024TFG: Unified Training-Free Guidance for Diffusion Modelscited by 107
  24. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 98
  25. 2023Generative Image Dynamicscited by 94
  26. 2025WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generationcited by 90
  27. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 88
  28. 2024PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generationcited by 80
  29. 2025Intuitive physics understanding emerges from self-supervised pretraining on natural videoscited by 70
  30. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 52
  31. 2025IntPhys 2: Benchmarking Intuitive Physics Understanding In Complex Synthetic Environmentscited by 52
  32. 2025Inference-time Scaling of Diffusion Models through Classical Searchcited by 49
  33. 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 41
  34. 2025Dynamic Search for Inference-Time Alignment in Diffusion Modelscited by 37
  35. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 34
  36. 2023Feedback-guided Data Synthesis for Imbalanced Classificationcited by 33
  37. 2023Prompt Expansion for Adaptive Text-to-Image Generationcited by 24
  38. 2024Improving Geo-diversity of Generated Images with Contextualized Vendi Score Guidancecited by 24

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 37.

37 papers.

Sort:

By area:

  1. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  2. 2026Solaris: Building a Multiplayer Video World Model in Minecraftcited by 47
  3. 2026Gamma-World: Generative Multi-Agent World Modeling Beyond Two Playerscited by 15
  4. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 11
  5. 2026Physics-IQ Verifiedcited by 9
  6. 2026PhyWorld: Physics-Faithful World Model for Video Generationcited by 6
  7. 2026Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generationcited by 6
  8. 2026Human Cognition in Machines: A Unified Perspective of World Modelscited by 5
  9. 2026PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignmentcited by 4
  10. 2026Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generationcited by 4
  11. 2026PhysOmni: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interactioncited by 4
  12. 2026MMPhysVideo: Physically Plausible Video Generation Through Joint RGB-Perception Modelingcited by 3
25 more
  1. 2026Physics in 2-Steps: Locking Motion Priors Before Visual Refinement Erases Themcited by 3
  2. 2026GEOPHYS: The Geometry of Physical Plausibilitycited by 3
  3. 2026CaliBench: Are the Stochastic Dynamics of Video World Models Physically Calibrated?cited by 3
  4. 2026Visko Orbis 1.0: A Live Model for Real-Time Interactive Long Video Generationcited by 2
  5. 2026Off-Manifold Refinement: Guiding Video Generators with a Frozen World Modelcited by 2
  6. 2026Why Do Video Diffusion Models Violate Physics? Unveiling the Flaws in Attention Mechanismscited by 2
  7. 2026Seeking Physics in Diffusion Noisecited by 1
  8. 2026MotiMotion: Motion-Controlled Video Generation with Visual Reasoningcited by 1
  9. 2026StressDream: Steering Video World Models for Robust Policy Evaluation and Improvementcited by 1
  10. 2026Physics-Informed Video Generation via Mixture-of-Experts Latent Alignmentcited by 1
  11. 2026PAVXploreRL: Physical-Action-Visual World Model Reinforcement Learning with Action Explorationcited by 1
  12. 2026SCOPE: Score-Isolated Agentic Optimization for Video World Modelscited by 0
  13. 2026Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioningcited by 0
  14. 2026Sampling headroom is not selection gain: a compute-value audit of test-time scaling for video world modelscited by 0
  15. 2026A Chosen Future Can Still Be Rewritten: Causal Writability in Video Modelscited by 0
  16. 2026Beyond Visual Quality: A Study of Test-Time Planning with World Action Modelscited by 0
  17. 2026Reimagine Video Dynamicscited by 0
  18. 2026PhyProbe: Rethinking Physical Consistency Evaluation in Generated Videoscited by 0
  19. 2026VR-JEPA: Learning Contrastive-State Latent Guidance for Generation-based Video Reasoningcited by 0
  20. 2026Video Generation Models: A Survey of Post-Training and Alignmentcited by 0
  21. 2026IntactWorld: Joint World Modeling with Intact Featurescited by 0
  22. 2026DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Trainingcited by 0
  23. 2026PhysForge: Material-Controllable 3D Reconstruction with Separable Componentscited by 0
  24. 2026World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Modelscited by 0
  25. 2026Multics: Generative Multi-Player World Model with Hierarchical Communicationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Improving the Physics of Video Generation with VJEPA-2 Reward Signal
  2. 2025TRAVL: A Recipe for Making Video-Language Models Better Judges of Physics Implausibility
  3. 2025ProPhy: Progressive Physical Alignment for Dynamic World Simulation
  4. 2025Bootstrapping Physics-Grounded Video Generation through VLM-Guided Iterative Self-Refinement
  5. 2026PhyGround: Benchmarking Physical Reasoning in Generative World Models
  6. 2026World Models' Last Exam in Physics
  7. 2026PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models
  8. 2026DiReCT: Disentangled Regularization of Contrastive Trajectories for Physics-Refined Video Generation