mlsanityData to 14 Oct 2026

Planning with Sketch-Guided Verification for Physics-Aware Video Generation

arXiv 2511.17450 ↗First posted 21 Nov 2025cs.CV cs.AI cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Recent video generation approaches increasingly rely on planning intermediate control signals such as object trajectories to improve temporal coherence and motion fidelity. However, these methods mostly employ single-shot plans that are typically limited to simple motions, or iterative refinement which requires multiple calls to the video generator, incuring high computational cost. To overcome these limitations, we propose SketchVerify, a training-free, sketch-verification-based planning framework that improves motion planning quality with more dynamically coherent trajectories (i.e., physically plausible and instruction-consistent motions) prior to full video generation by introducing a test-time sampling and verification loop. Given a prompt and a reference image, our method predicts multiple candidate motion plans and ranks them using a vision-language verifier that jointly evaluates semantic alignment with the instruction and physical plausibility. To efficiently score candidate motion plans, we render each trajectory as a lightweight video sketch by compositing objects over a static background, which bypasses the need for expensive, repeated diffusion-based synthesis while achieving comparable performance. We iteratively refine the motion plan until a satisfactory one is identified, which is then passed to the trajectory-conditioned generator for final synthesis. Experiments on WorldModelBench and PhyWorldBench demonstrate that our method significantly improves motion quality, physical realism, and long-term consistency compared to competitive baselines while being substantially more efficient. Our ablation study further shows that scaling up the number of trajectory candidates consistently enhances overall performance.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 44 of its 50 references that are in the map. First posted on arXiv 21 Nov 2025.

Builds on

44 of 50 references.

References · 44

  1. 2023Segment Anythingcited by 8,234
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  3. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  4. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  7. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  8. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  9. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 535
  10. 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 247
  11. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  12. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
32 more
  1. 2025Do generative video models understand physical principles?cited by 161
  2. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 149
  3. 2023VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planningcited by 141
  4. 2025WorldModelBench: Judging Video Generation Models As World Modelscited by 124
  5. 2023Animate-A-Story: Storytelling with Retrieval-Augmented Video Generationcited by 90
  6. 2023LLM-grounded Video Diffusion Modelscited by 83
  7. 2024PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generationcited by 80
  8. 2024Vlogger: Make Your Dream A Vlogcited by 70
  9. 2025Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signalscited by 53
  10. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 52
  11. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generationcited by 50
  12. 2025PISA Experiments: Exploring Physics Post-Training for Video Diffusion Models by Watching Stuff Dropcited by 50
  13. 2024MotionCraft: Physics-based Zero-Shot Video Generationcited by 48
  14. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  15. 2025ATI: Any Trajectory Instruction for Controllable Video Generationcited by 43
  16. 2025PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generationcited by 41
  17. 2025VEGGIE: Instructional Editing and Reasoning Video Concepts with Grounded Generationcited by 40
  18. 2024LeviTor: 3D Trajectory Oriented Image-to-Video Synthesiscited by 39
  19. 2025OmniEraser: Remove Objects and Their Effects in Images with Paired Video-Frame Datacited by 37
  20. 2025"PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Modelscited by 36
  21. 2025VChain: Chain-of-Visual-Thought for Reasoning in Video Generationcited by 32
  22. 2025VISTA: A Test-Time Self-Improving Video Generation Agentcited by 31
  23. 2025Step-Video-TI2V Technical Report: A State-of-the-Art Text-Driven Image-to-Video Generation Modelcited by 28
  24. 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 21
  25. 2025Reasoning Physical Video Generation with Diffusion Timestep Tokens via Reinforcement Learningcited by 18
  26. 2025PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selectioncited by 15
  27. 2024Self-Correcting Text-to-Video Generation with Misalignment Detection and Localized Refinementcited by 14
  28. 2025Enhancing Physical Plausibility in Video Generation by Reasoning the Implausibilitycited by 13
  29. 2021Click to Move: Controlling Video Generation with Sparse Motioncited by 11
  30. 2025Frame Guidance: Training-Free Guidance for Frame-Level Control in Video Diffusion Modelscited by 11
  31. 2025Training-free Guidance in Text-to-Video Generation via Multimodal Planning and Structured Noise Initializationcited by 9
  32. 2024RACCooN: A Versatile Instructional Video Editing Framework with Auto-Generated Narrativescited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.