AcrossVAM2.0: Conflict-Aware Point Supervision for Parameter-Efficient Video Diffusion Adaptation
On the map: Diffusion models → Video generation
Abstract (arXiv)
Explicit point correspondence is attractive for adapting video generators: it describes how visible surfaces move, but can overwrite the appearance prior learned by a large pretrained model. We study this tension in a 5.01B-parameter text-to-video diffusion transformer. ACROSSVAM2.0 combines a confidence-weighted track-and-visibility head, zero-initialized residual feedback, and asymmetric PCGrad that removes only the auxiliary component opposing the primary flow-matching gradient on shared LoRA parameters. The design preserves the generator exactly at initialization and protects its primary update during adaptation. In a four-way ablation trained for 1,000 steps with three independent seeds, the full method lowers KLT forward-backward cycle error in all three seeds relative to a matched flow-only continuation (mean seed-level difference -0.00711, SD 0.00505), while robust sharpness remains positive in all three. An expanded 24-prompt evaluation (216 videos) yields 9.7% lower aggregate CoTracker3 cycle error than the matched control and near-matched performance on six VBench dimensions; however, the seed-level tracking effect is heterogeneous and I3D-FVD regresses from 0.1824 to 0.2022. These results support a bounded claim: conflict-aware adaptation improves correspondence consistency without broad perceptual degradation, but does not uniformly improve video quality. We release equations, pseudocode, matched outputs, source tables, and artifact provenance.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 26 of its 32 references that are in the map. First posted on arXiv 10 Oct 2026.
Builds on
26 of 32 references.
Closest ancestors · 5
Same direction, newest first
- 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
- 2023ModelScope Text-to-Video Technical Report12 Aug
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
- 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
Other references · 21
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2020Gradient Surgery for Multi-Task Learningcited by 1,256
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
9 more
- 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
- 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
- 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
- 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 287
- 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 263
- 2022Multi-Task Learning as a Bargaining Gamecited by 215
- 2023Tracking Everything Everywhere All at Oncecited by 193
- 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 178
- 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 120
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Not All Tokens Need 40 Steps: Heterogeneous Step Allocation in Diffusion Transformers for Efficient Video Generation
- 2026LiteVSR: Lightweight Adaptation of Frozen Diffusion Transformers for Video Super-Resolution
- 2026LoopREG: Representation Supervision Enables Test-Time Recurrent Depth in Diffusion Transformers
- 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generation
- 2025Emergent Temporal Correspondences from Video Diffusion Transformers
- 2024Towards Precise Scaling Laws for Video Diffusion Transformers
- 2026DART: Distillation-Aware Reparameterization for Training-Free LoRA Reuse in Few-Step Video Diffusion Models
- 2025Decouple and Track: Benchmarking and Improving Video Diffusion Transformers for Motion Transfer