Planning with Sketch-Guided Verification for Physics-Aware Video Generation
Not on the map. Why?
Abstract (arXiv)
Recent video generation approaches increasingly rely on planning intermediate control signals such as object trajectories to improve temporal coherence and motion fidelity. However, these methods mostly employ single-shot plans that are typically limited to simple motions, or iterative refinement which requires multiple calls to the video generator, incuring high computational cost. To overcome these limitations, we propose SketchVerify, a training-free, sketch-verification-based planning framework that improves motion planning quality with more dynamically coherent trajectories (i.e., physically plausible and instruction-consistent motions) prior to full video generation by introducing a test-time sampling and verification loop. Given a prompt and a reference image, our method predicts multiple candidate motion plans and ranks them using a vision-language verifier that jointly evaluates semantic alignment with the instruction and physical plausibility. To efficiently score candidate motion plans, we render each trajectory as a lightweight video sketch by compositing objects over a static background, which bypasses the need for expensive, repeated diffusion-based synthesis while achieving comparable performance. We iteratively refine the motion plan until a satisfactory one is identified, which is then passed to the trajectory-conditioned generator for final synthesis. Experiments on WorldModelBench and PhyWorldBench demonstrate that our method significantly improves motion quality, physical realism, and long-term consistency compared to competitive baselines while being substantially more efficient. Our ablation study further shows that scaling up the number of trajectory candidates consistently enhances overall performance.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 44 of its 50 references that are in the map. First posted on arXiv 21 Nov 2025.
Builds on
44 of 50 references.
References · 44
- 2023Segment Anythingcited by 8,234
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 535
- 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 247
- 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
- 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
32 more
- 2025Do generative video models understand physical principles?cited by 161
- 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 149
- 2023VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planningcited by 141
- 2025WorldModelBench: Judging Video Generation Models As World Modelscited by 124
- 2023Animate-A-Story: Storytelling with Retrieval-Augmented Video Generationcited by 90
- 2023LLM-grounded Video Diffusion Modelscited by 83
- 2024PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generationcited by 80
- 2024Vlogger: Make Your Dream A Vlogcited by 70
- 2025Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signalscited by 53
- 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 52
- 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generationcited by 50
- 2025PISA Experiments: Exploring Physics Post-Training for Video Diffusion Models by Watching Stuff Dropcited by 50
- 2024MotionCraft: Physics-based Zero-Shot Video Generationcited by 48
- 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
- 2025ATI: Any Trajectory Instruction for Controllable Video Generationcited by 43
- 2025PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generationcited by 41
- 2025VEGGIE: Instructional Editing and Reasoning Video Concepts with Grounded Generationcited by 40
- 2024LeviTor: 3D Trajectory Oriented Image-to-Video Synthesiscited by 39
- 2025OmniEraser: Remove Objects and Their Effects in Images with Paired Video-Frame Datacited by 37
- 2025"PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Modelscited by 36
- 2025VChain: Chain-of-Visual-Thought for Reasoning in Video Generationcited by 32
- 2025VISTA: A Test-Time Self-Improving Video Generation Agentcited by 31
- 2025Step-Video-TI2V Technical Report: A State-of-the-Art Text-Driven Image-to-Video Generation Modelcited by 28
- 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 21
- 2025Reasoning Physical Video Generation with Diffusion Timestep Tokens via Reinforcement Learningcited by 18
- 2025PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selectioncited by 15
- 2024Self-Correcting Text-to-Video Generation with Misalignment Detection and Localized Refinementcited by 14
- 2025Enhancing Physical Plausibility in Video Generation by Reasoning the Implausibilitycited by 13
- 2021Click to Move: Controlling Video Generation with Sparse Motioncited by 11
- 2025Frame Guidance: Training-Free Guidance for Frame-Level Control in Video Diffusion Modelscited by 11
- 2025Training-free Guidance in Text-to-Video Generation via Multimodal Planning and Structured Noise Initializationcited by 9
- 2024RACCooN: A Versatile Instructional Video Editing Framework with Auto-Generated Narrativescited by 8
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoningcited by 14
- 2026A2RD: Agentic Autoregressive Diffusion for Long Video Consistencycited by 6
- 2026PhyMotion: Structured 3D Motion Reward for Physics-Grounded Human Video Generationcited by 3
- 2026Physics Question Scene Graph: Fine-grained Evaluation of Physical Plausibility in Text-to-Video Generationcited by 2
- 2026Sampling headroom is not selection gain: a compute-value audit of test-time scaling for video world modelscited by 0
- 2026HiPhy: Hierarchical Alignment for Physically-Plausible Multi-Principle Video Generationcited by 0
- 2026HiPhy: Hierarchical Alignment for Physically-Plausible Multi-Principle Video Generation1 Oct 2026
- 2026Sampling headroom is not selection gain: a compute-value audit of test-time scaling for video world models6 Sep 2026
- 2026Physics Question Scene Graph: Fine-grained Evaluation of Physical Plausibility in Text-to-Video Generation24 Jun 2026
- 2026PhyMotion: Structured 3D Motion Reward for Physics-Grounded Human Video Generation14 May 2026
- 2026A2RD: Agentic Autoregressive Diffusion for Long Video Consistency7 May 2026
- 2026When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoning9 Feb 2026