# Physics-Aware Video Generation via Agentic Planning and Graph-Guided Optimization

arXiv: https://arxiv.org/abs/2609.13006 · HTML: https://mlsanity.com/paper/2609.13006/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 36 of its 41 references that are in the map.
- First posted on arXiv 16 Jul 2026.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)
- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 30 of 36)

- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [Open-Sora: Democratizing Efficient Video Production for All](https://mlsanity.com/paper/2412.20404/) (2024)
- [LTX-Video: Realtime Video Latent Diffusion](https://mlsanity.com/paper/2501.00103/) (2024)
- [Universal Guidance for Diffusion Models](https://mlsanity.com/paper/2302.07121/) (2023)
- [LayoutGPT: Compositional Visual Planning and Generation with Large Language Models](https://mlsanity.com/paper/2305.15393/) (2023)
- [FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model](https://mlsanity.com/paper/2303.09833/) (2023)
- [How Far is Video Generation from World Model: A Physical Law Perspective](https://mlsanity.com/paper/2411.02385/) (2024)
- [PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation](https://mlsanity.com/paper/2409.18964/) (2024)
- [Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation](https://mlsanity.com/paper/2410.05363/) (2024)
- [Do generative video models understand physical principles?](https://mlsanity.com/paper/2501.09038/) (2025)
- [PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding](https://mlsanity.com/paper/2501.16411/) (2025)
- [VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning](https://mlsanity.com/paper/2309.15091/) (2023)
- [LLM-grounded Video Diffusion Models](https://mlsanity.com/paper/2309.17444/) (2023)
- [PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation](https://mlsanity.com/paper/2412.00596/) (2024)
- [Kling-Omni Technical Report](https://mlsanity.com/paper/2512.16776/) (2025)
- [On the Content Bias in Fr'echet Video Distance](https://mlsanity.com/paper/2404.12391/) (2024)
- [VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Prior](https://mlsanity.com/paper/2503.23368/) (2025)
- [Understanding and Improving Training-free Loss-based Diffusion Guidance](https://mlsanity.com/paper/2403.12404/) (2024)
- [VChain: Chain-of-Visual-Thought for Reasoning in Video Generation](https://mlsanity.com/paper/2510.05094/) (2025)
- [PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation](https://mlsanity.com/paper/2512.24551/) (2025)
- [Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation](https://mlsanity.com/paper/2508.10858/) (2025)
- [QuantiPhy: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Models](https://mlsanity.com/paper/2512.19526/) (2025)

## Built on by (most cited first, 1 of 1)

- [Phase-aware video generation for physics-grounded dynamics and interactions](https://mlsanity.com/paper/2610.11791/) (2026)

## Abstract (arXiv)

Video diffusion models (VDMs) synthesize photorealistic content, yet they often fail to follow the course that a physical phenomenon should take within a given scene. Recent training-free methods let a vision-language model (VLM) plan the phenomenon and guide a frozen VDM toward the plan; however, such plans are derived from the prompt and consumed as whole keyframes or trajectories, which leaves unspecified where the consequences land in the observed scene and turns incidental visual details into optimization targets. We observe that a phenomenon specified in words unfolds as sparse, local changes to the physical state of the observed scene. Building on this observation, we present PhysPlan, a training-free image-to-video framework that represents a phenomenon as a grounded state graph and uses this graph to decide what, where, and when the guidance constrains. Grounded Physical State Reasoning decomposes the phenomenon into physical deltas, each stating which objects change, to what state, and by which physical rule, and translates each delta into graph edits, verified by deterministic checks, that leave all other objects unchanged. Graph-Guided Test-Time Optimization renders a keyframe for each state, measures the denoised estimates only along the properties selected by the edits, and concentrates the update on the edited objects. On PhyGenBench and Physics-IQ, PhysPlan raises its base model from 0.52 to 0.77 and from 27.1 to 38.2, surpassing the strongest prior I2V method (0.60 and 34.6), and lowers FVD by over 20%. Project page: https://physplan.github.io