# KD-CVG: A Knowledge-Driven Approach for Creative Video Generation

arXiv: https://arxiv.org/abs/2604.21362 · HTML: https://mlsanity.com/paper/2604.21362/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 22 references that are in the map.
- First posted on arXiv 23 Apr 2026.

## Builds on (most cited first, 19 of 19)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [Open-Sora: Democratizing Efficient Video Production for All](https://mlsanity.com/paper/2412.20404/) (2024)
- [VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models](https://mlsanity.com/paper/2401.09047/) (2024)
- [Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2309.15818/) (2023)
- [CogVLM2: Visual Language Models for Image and Video Understanding](https://mlsanity.com/paper/2408.16500/) (2024)
- [ProPainter: Improving Propagation and Transformer for Video Inpainting](https://mlsanity.com/paper/2309.03897/) (2023)
- [Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations](https://mlsanity.com/paper/2410.10792/) (2024)
- [Sample Efficient Reinforcement Learning with REINFORCE](https://mlsanity.com/paper/2010.11364/) (2020)
- [VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models](https://mlsanity.com/paper/2312.00845/) (2023)
- [A New Creative Generation Pipeline for Click-Through Rate with Stable Diffusion Model](https://mlsanity.com/paper/2401.10934/) (2024)
- [Planning and Rendering: Towards Product Poster Generation with Diffusion Models](https://mlsanity.com/paper/2312.08822/) (2023)
- [Towards Reliable Advertising Image Generation Using Human Feedback](https://mlsanity.com/paper/2408.00418/) (2024)
- [CTR-Driven Advertising Image Generation with Multimodal Large Language Models](https://mlsanity.com/paper/2502.06823/) (2025)
- [Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluation](https://mlsanity.com/paper/2508.02374/) (2025)
- [xGen-VideoSyn-1: High-fidelity Text-to-Video Synthesis with Compressed Representations](https://mlsanity.com/paper/2408.12590/) (2024)

## Abstract (arXiv)

Creative Generation (CG) leverages generative models to automatically produce advertising content that highlights product features, and it has been a significant focus of recent research. However, while CG has advanced considerably, most efforts have concentrated on generating advertising text and images, leaving Creative Video Generation (CVG) relatively underexplored. This gap is largely due to two major challenges faced by Text-to-Video (T2V) models: (a) ambiguous semantic alignment, where models struggle to accurately correlate product selling points with creative video content, and (b) inadequate motion adaptability, resulting in unrealistic movements and distortions. To address these challenges, we develop a comprehensive Advertising Creative Knowledge Base (ACKB) as a foundational resource and propose a knowledge-driven approach (KD-CVG) to overcome the knowledge limitations of existing models. KD-CVG consists of two primary modules: Semantic-Aware Retrieval (SAR) and Multimodal Knowledge Reference (MKR). SAR utilizes the semantic awareness of graph attention networks and reinforcement learning feedback to enhance the model's comprehension of the connections between selling points and creative videos. Building on this, MKR incorporates semantic and motion priors into the T2V model to address existing knowledge gaps. Extensive experiments have demonstrated KD-CVG's superior performance in achieving semantic alignment and motion adaptability, validating its effectiveness over other state-of-the-art methods. The code and dataset will be open source at https://kdcvg.github.io/KDCVG/.