mlsanityData to 14 Oct 2026

G3-LoRA: Organizing Reward-Weighted Video Data with Gradient-Guided Grouped LoRA

arXiv 2609.35189 ↗First posted 28 Sep 2026cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-training foundation video models on heterogeneous reward-weighted data usually assume that all data categories induce compatible updates. This assumption is fragile when categories correspond to different skills, domains, or evaluation dimensions. We study this problem in text-to-video post-training, where VBench2.0 dimensions define data buckets and an external multimodal reward pipeline assigns sample weights. We propose G3-LoRA (Gradient-Guided Grouped LoRA), a data organization procedure that probes category-level gradients induced by reward-weighted video samples, removes the shared global update direction, clusters categories by residual gradient compatibility, trains group-specific LoRA experts, and consolidates them into one adapter by weight merging followed by on-policy distillation from the experts. We motivate this procedure by viewing reward-weighted flow matching as velocity-field regression: incompatible reward dimensions may prefer different denoising directions in overlapping noisy latent regions, causing shared LoRA training to average capabilities. On Wan2.1-T2V-1.3B-Diffusers, the merged grouped adapter improves the matched VBench2.0 evaluation over the base model, a joint reward-weighted LoRA baseline, and random, semantic, and raw-gradient partitions trained with the same pipeline; an independent evaluator agrees, and on CogVideoX-2B grouping avoids the negative transfer of joint training. The gain is not uniform: merging compresses the largest specialist gains, distillation recovers part of this loss, and camera motion and several local-quality dimensions remain challenging. Together, these results suggest that gradient compatibility can serve as a practical diagnostic for organizing reward-weighted video post-training data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 41 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

39 of 41 references.

References · 39

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2022Flow Matching for Generative Modelingcited by 5,702
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  8. 2025Qwen3-VL Technical Reportcited by 2,793
  9. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  10. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  12. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
27 more
  1. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  2. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  3. 2022Editing Models with Task Arithmeticcited by 1,120
  4. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  5. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  6. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  7. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  8. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  9. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  10. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  11. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  12. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  13. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  14. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 288
  15. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  16. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
  17. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
  18. 2020Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Modelscited by 169
  19. 2024Zigzag Diffusion Sampling: Diffusion Models Can Self-Improve via Self-Reflectioncited by 41
  20. 2024Alignment of Diffusion Models: Fundamentals, Challenges, and Futurecited by 35
  21. 2026Flow-OPD: On-Policy Distillation for Flow Matching Modelscited by 34
  22. 2025Weak-to-Strong Diffusion with Reflectioncited by 13
  23. 2026Optimizing Few-Step Generation with Adaptive Matching Distillationcited by 11
  24. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 7
  25. 2026MaineCoon: Pursuing A Real-Time Audio-Visual Social World Modelcited by 6
  26. 2026CRAFT: Aligning Diffusion Models with Fine-Tuning Is Easier Than You Thinkcited by 4
  27. 2026Exploring Data-Free LoRA Transferability for Video Diffusion Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Video Diffusion Alignment via Reward Gradients
  2. 2026GDRO: Group-level Reward Post-training Suitable for Diffusion Models
  3. 2025Rethinking Reward Signals in Video GRPO: When Scores Become Targets
  4. 2026CoRe: Co-Evolving Reward Models for Mitigating Latent Reward Hacking in Video Diffusion Models
  5. 2026Refining Multidimensional Video Reward Models via Disentangled Influence Functions
  6. 2025SoliReward: Mitigating Susceptibility to Reward Hacking and Annotation Noise in Video Generation Reward Models
  7. 2026Prompt2Effect: Training-Free Image-to-Video Model Specialization via LoRA Generation
  8. 2026Diffusion-DRF: Free, Rich, and Differentiable Reward for Video Diffusion Fine-Tuning