# Phantom: Physics-Infused Video Generation via Joint Modeling of Visual and Latent Physical Dynamics

arXiv: https://arxiv.org/abs/2604.08503 · HTML: https://mlsanity.com/paper/2604.08503/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 48 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models](https://mlsanity.com/paper/2401.09047/) (2024)
- [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023)
- [Lumiere: A Space-Time Diffusion Model for Video Generation](https://mlsanity.com/paper/2401.12945/) (2024)
- [LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models](https://mlsanity.com/paper/2309.15103/) (2023)
- [OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation](https://mlsanity.com/paper/2407.02371/) (2024)
- [VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness](https://mlsanity.com/paper/2503.21755/) (2025)
- [Flow Matching on General Geometries](https://mlsanity.com/paper/2302.03660/) (2023)
- [VideoPhy: Evaluating Physical Commonsense for Video Generation](https://mlsanity.com/paper/2406.03520/) (2024)
- [How Far is Video Generation from World Model: A Physical Law Perspective](https://mlsanity.com/paper/2411.02385/) (2024)
- [PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation](https://mlsanity.com/paper/2409.18964/) (2024)
- [Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation](https://mlsanity.com/paper/2410.05363/) (2024)
- [VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation](https://mlsanity.com/paper/2503.06800/) (2025)
- [VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models](https://mlsanity.com/paper/2502.02492/) (2025)
- [VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Models](https://mlsanity.com/paper/2505.23656/) (2025)

## Built on by (most cited first, 12 of 12)

- [NEWTON: Agentic Planning for Physically Grounded Video Generation](https://mlsanity.com/paper/2605.18396/) (2026)
- [LaMo: Self-Supervised Latent Motion Priors for Physical Realism in Video Generation](https://mlsanity.com/paper/2605.23878/) (2026)
- [TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Image](https://mlsanity.com/paper/2609.04911/) (2026)
- [PhysWeep: Does a Video Generator Realize the Physics You Ask For?](https://mlsanity.com/paper/2609.06207/) (2026)
- [Physics-Informed Video Generation via Mixture-of-Experts Latent Alignment](https://mlsanity.com/paper/2606.04737/) (2026)
- [Neural Voxel Dynamics: Learning Volumetric Feature Advection for 3D Physics in V-JEPA Latent Space](https://mlsanity.com/paper/2606.26410/) (2026)
- [GraphVid: Interactive Graph-Controllable Video Generation](https://mlsanity.com/paper/2607.21580/) (2026)
- [VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generation](https://mlsanity.com/paper/2607.23472/) (2026)
- [Diagnosing Under-Development of Irreversible Processes in Video Generation](https://mlsanity.com/paper/2608.00617/) (2026)
- [Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioning](https://mlsanity.com/paper/2609.00656/) (2026)
- [World Models' Last Exam in Physics](https://mlsanity.com/paper/2610.08791/) (2026)
- [World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models](https://mlsanity.com/paper/2610.16365/) (2026)

## Abstract (arXiv)

Recent advances in generative video modeling, driven by large-scale datasets and powerful architectures, have yielded remarkable visual realism. However, emerging evidence suggests that simply scaling data and model size does not endow these systems with an understanding of the underlying physical laws that govern real-world dynamics. Existing approaches often fail to capture or enforce such physical consistency, resulting in unrealistic motion and dynamics. In his work, we investigate whether integrating the inference of latent physical properties directly into the video generation process can equip models with the ability to produce physically plausible videos. To this end, we propose Phantom, a Physics-Infused Video Generation model that jointly models the visual content and latent physical dynamics. Conditioned on observed video frames and inferred physical states, Phantom jointly predicts latent physical dynamics and generates future video frames. Phantom leverages a physics-aware video representation that serves as an abstract yet informaive embedding of the underlying physics, facilitating the joint prediction of physical dynamics alongside video content without requiring an explicit specification of a complex set of physical dynamics and properties. By integrating the inference of physical-aware video representation directly into the video generation process, Phantom produces video sequences that are both visually realistic and physically consistent. Quantitative and qualitative results on both standard video generation and physics-aware benchmarks demonstrate that Phantom not only outperforms existing methods in terms of adherence to physical dynamics but also delivers competitive perceptual fidelity.