mlsanityData to 7 Oct 2026

T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Design

arXiv 2410.05677 ↗First posted 8 Oct 2024cs.CV cs.AIcited by 54

On the map: Diffusion models → Video generation

also in 1 more
Abstract (arXiv)

In this paper, we focus on enhancing a diffusion-based text-to-video (T2V) model during the post-training phase by distilling a highly capable consistency model from a pretrained T2V model. Our proposed method, T2V-Turbo-v2, introduces a significant advancement by integrating various supervision signals, including high-quality training data, reward model feedback, and conditional guidance, into the consistency distillation process. Through comprehensive ablation studies, we highlight the crucial importance of tailoring datasets to specific learning objectives and the effectiveness of learning from diverse reward models for enhancing both the visual quality and text-video alignment. Additionally, we highlight the vast design space of conditional guidance strategies, which centers on designing an effective energy function to augment the teacher ODE solver. We demonstrate the potential of this approach by extracting motion guidance from the training datasets and incorporating it into the ODE solver, showcasing its effectiveness in improving the motion quality of the generated videos with the improved motion-related metrics from VBench and T2V-CompBench. Empirically, our T2V-Turbo-v2 establishes a new state-of-the-art result on VBench, with a Total score of 85.13, surpassing proprietary systems such as Gen-3 and Kling.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 65 of its 73 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

65 of 73 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024Video Diffusion Alignment via Reward Gradients11 Jul
  2. 2024T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback29 May
  3. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
  4. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedback19 Dec
  5. 2023VideoLCM: Video Latent Consistency Model14 Dec
  6. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generation30 Oct
  7. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation27 Sep
  8. 2023ModelScope Text-to-Video Technical Report12 Aug
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  10. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  11. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators23 Mar
  12. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
2 more
  1. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  2. 2022Video Diffusion Models7 Apr

Other references · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
38 more
  1. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  2. 2020Learning to summarize from human feedbackcited by 2,562
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  5. 2023Consistency Modelscited by 1,778
  6. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  7. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  8. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  9. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  10. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  11. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  12. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  13. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  14. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  15. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  16. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  17. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  18. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  19. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
  20. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  21. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 284
  22. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  23. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 280
  24. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  25. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 190
  26. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  27. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 177
  28. 2022EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equationscited by 172
  29. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
  30. 2023Localizing Object-level Shape Variations with Text-to-Image Diffusion Modelscited by 142
  31. 2022GRiT: A Generative Region-to-text Transformer for Object Understandingcited by 119
  32. 2024MotionClone: Training-Free Motion Cloning for Controllable Video Generationcited by 104
  33. 2023Expressive Text-to-Image Generation with Rich Textcited by 89
  34. 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Conditioncited by 89
  35. 2023Tag2Text: Guiding Vision-Language Model via Image Taggingcited by 84
  36. 2024Large-scale Reinforcement Learning for Diffusion Modelscited by 78
  37. 2024Vript: A Video Is Worth Thousands of Wordscited by 71
  38. 2024VidGen-1M: A Large-Scale Dataset for Text-to-video Generationcited by 60

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 31; 2026: 13.

54 papers.

Sort:

By area:

  1. 2025Improving Video Generation with Human Feedbackcited by 267
  2. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 194
  3. 2025WorldModelBench: Judging Video Generation Models As World Modelscited by 116
  4. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 102
  5. 2025Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistencycited by 81
  6. 2024LiFT: Leveraging Human Feedback for Text-to-Video Model Alignmentcited by 51
  7. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 48
  8. 2024LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexitycited by 41
  9. 2025DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Modelscited by 32
  10. 2024SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Devicecited by 25
  11. 2024DOLLAR: Few-Step Video Generation via Distillation and Latent Reward Optimizationcited by 25
  12. 2024AV-Link: Temporally-Aligned Diffusion Features for Cross-Modal Audio-Video Generationcited by 21
38 more
  1. 2025MagicDistillation: Weak-to-Strong Video Distillation for Large-Scale Few-Step Synthesiscited by 18
  2. 2024InstanceCap: Improving Text-to-Video Generation via Instance-aware Structured Captioncited by 17
  3. 2025T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Modelscited by 17
  4. 2025MagicInfinite: Generating Infinite Talking Videos with Your Words and Voicecited by 16
  5. 2025Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMscited by 16
  6. 2026Flash-WAM: Modality-Aware Distillation for World Action Modelscited by 16
  7. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  8. 2025PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selectioncited by 14
  9. 2025AccVideo: Accelerating Video Diffusion Model with Synthetic Datasetcited by 13
  10. 2025Dual-IPO: Dual-Iterative Preference Optimization for Text-to-Video Generationcited by 12
  11. 2024Neuro-Symbolic Evaluation of Text-to-Video Models using Formal Verificationcited by 9
  12. 2025ImagerySearch: Adaptive Test-Time Search for Video Generation Beyond Semantic Dependency Constraintscited by 9
  13. 2024Individual Content and Motion Dynamics Preserved Pruning for Video Diffusion Modelscited by 8
  14. 2025AMD-Hummingbird: Towards an Efficient Text-to-Video Modelcited by 8
  15. 2025Aligning Anime Video Generation with Human Feedbackcited by 8
  16. 2026Diffusion-DRF: Free, Rich, and Differentiable Reward for Video Diffusion Fine-Tuningcited by 8
  17. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 6
  18. 2025ShortFT: Diffusion Model Alignment via Shortcut-based Fine-Tuningcited by 5
  19. 2025Harness Local Rewards for Global Benefits: Effective Text-to-Video Generation Alignment with Patch-level Reward Modelscited by 4
  20. 2025VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videoscited by 4
  21. 2025TTOM: Test-Time Optimization and Memorization for Compositional Video Generationcited by 4
  22. 2025AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Pathcited by 4
  23. 2026FastLightGen: Fast and Light Video Generation with Fewer Steps and Parameterscited by 4
  24. 2026Flex-Forcing: Towards a Unified Autoregressive and Bidirectional Video Diffusion Modelcited by 4
  25. 2024Motion by Queries: Identity-Motion Trade-offs in Text-to-Video Generationcited by 3
  26. 2026VERTIGO: Visual Preference Optimization for Cinematic Camera Trajectory Generationcited by 3
  27. 2025Respond Beyond Language: A Benchmark for Video Generation in Response to Realistic User Intentscited by 2
  28. 2025Causal-Entity Reflected Egocentric Traffic Accident Video Synthesiscited by 2
  29. 2025Reg-DPO: SFT-Regularized Direct Preference Optimization with GT-Pair for Improving Video Generationcited by 2
  30. 2025IRPO: Boosting Image Restoration via Post-training GRPOcited by 2
  31. 2026GPD: Guided Progressive Distillation for Fast and High-Quality Video Generationcited by 2
  32. 2026Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Modelcited by 2
  33. 2025Minute-Long Videos with Dual Parallelismscited by 1
  34. 2025When Distillation Breaks Motion Control: Restoring Generative Trajectories for Fast Video Generatorscited by 1
  35. 2025Generative AI Meets 6G and Beyond: Diffusion Models for Semantic Communicationscited by 1
  36. 2026Training-free Motion Factorization for Compositional Video Generationcited by 1
  37. 2026DiReCT: Disentangled Regularization of Contrastive Trajectories for Physics-Refined Video Generationcited by 1
  38. 2025Comp-Attn: Present-and-Align Attention for Compositional Video Generationcited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation
  2. 2026HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models
  3. 2024Searching Priors Makes Text-to-Video Synthesis Better
  4. 2025V.I.P. : Iterative Online Preference Distillation for Efficient Video Diffusion Models
  5. 2026VDOT++: Unified Few-Step Video Generation via Unbalanced Optimal Transport Distillation
  6. 2023Control-A-Video: Controllable Text-to-Video Diffusion Models with Motion Prior and Reward Feedback Learning
  7. 2025Training-free Guidance in Text-to-Video Generation via Multimodal Planning and Structured Noise Initialization
  8. 2026A Systematic Post-Train Framework for Video Generation