mlsanityData to 7 Oct 2026

Enhancing Video Physical Consistency via Role-aware Joint Training and Modality-decoupled Denoising

arXiv 2607.04653 ↗First posted 6 Jul 2026cs.CVcited by 2

On the map: Diffusion models → Video generation

Abstract (arXiv)

While modern video diffusion models excel in visual fidelity, maintaining long-range physical consistency remains a formidable challenge. Conventional pixel-reconstruction objectives mainly focus on appearance details and often fail to capture the underlying dynamics of a scene. To mitigate this, recent efforts have integrated auxiliary modalities (e.g., optical flow) to introduce physics priors via joint training with video appearance. However, these methods have three main limitations: (1) they do not distinguish the different motion patterns of different entity types; (2) joint modeling of visual and auxiliary modalities can cause capacity conflicts and weaken the pretrained visual prior; and (3) auxiliary modalities may accumulate errors during inference. To address these issues, we propose VPT, a fine-tuning framework for improving physical consistency in video diffusion models. VPT introduces a role-aware signal that groups entities into agents, controlled objects, passive objects, and background, so that different physical roles can be modeled more clearly. We further propose a modality-decoupled denoising strategy, where the visual and auxiliary channels are assigned independent noise levels. Together with a loss-weight decay strategy, this design makes auxiliary modalities serve as soft constraints rather than strong dependencies, mitigating recursive prediction errors during inference. We also introduce cross-step auto-guidance to further strengthen physical dynamics. Experiments show that VPT improves physical consistency while preserving visual quality, achieving relative gains of 39.4% in SA and 17.9% in PC on VideoPhy benchmark over Wan2.1-T2V-1.3B, and consistent improvements on VideoPhy-2 benchmark. The project page is available at https://tom-zgt.github.io/VPT.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 26 of its 34 references that are in the map. First posted on arXiv 6 Jul 2026.

Builds on

26 of 34 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Models29 May
  2. 2025OmniVDiff: Omni Controllable Video Diffusion for Generation and Understanding15 Apr
  3. 2025Goku: Flow Based Video Generative Foundation Models7 Feb
  4. 2024Autoregressive Video Generation without Vector Quantization18 Dec
  5. 2024PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation30 Nov
  6. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  7. 2024Latte: Latent Diffusion Transformer for Video Generation5 Jan
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  10. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  11. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  12. 2022Video Diffusion Models7 Apr

Other references · 14

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  6. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  7. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  8. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 286
  9. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
  10. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 169
  11. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 141
  12. 2025InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generationcited by 43
2 more
  1. 20254DNeX: Feed-Forward 4D Generative Modeling Made Easycited by 31
  2. 2025UnityVideo: Unified Multi-Modal Multi-Task Learning for Enhancing World-Aware Video Generationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025LINA: Learning INterventions Adaptively for Physical Alignment and Generalization in Diffusion Models
  2. 2026SIFT: Self-Imagination Fine-Tuning for Physically Plausible Motion in Video Diffusion Models
  3. 2026The Invisible Hand of Physics: When Video Diffusion Models Know More Than They Show
  4. 2026V-Co: A Closer Look at Visual Representation Alignment via Co-Denoising
  5. 2025Temporal-Consistent Video Restoration with Pre-trained Diffusion Models
  6. 2025Think Before You Diffuse: Infusing Physical Rules into Video Diffusion
  7. 2024Temporal Residual Guided Diffusion Framework for Event-Driven Video Reconstruction
  8. 2026Physics in 2-Steps: Locking Motion Priors Before Visual Refinement Erases Them