# VPO: Aligning Text-to-Video Generation Models with Prompt Optimization

arXiv: https://arxiv.org/abs/2503.20491 · HTML: https://mlsanity.com/paper/2503.20491/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 26 of its 30 references that are in the map.
- First posted on arXiv 26 Mar 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [Diffusion Model Alignment Using Direct Preference Optimization](https://mlsanity.com/paper/2311.12908/) (2023)
- [Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators](https://mlsanity.com/paper/2303.13439/) (2023)
- [VideoCrafter1: Open Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2310.19512/) (2023)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [Optimizing Prompts for Text-to-Image Generation](https://mlsanity.com/paper/2212.09611/) (2022)
- [VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation](https://mlsanity.com/paper/2412.21059/) (2024)
- [VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Models](https://mlsanity.com/paper/2403.06098/) (2024)
- [Black-Box Prompt Optimization: Aligning Large Language Models without Model Training](https://mlsanity.com/paper/2311.04155/) (2023)
- [T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models](https://mlsanity.com/paper/2407.05965/) (2024)
- [Prompt-A-Video: Prompt Your Video Diffusion Model via Preference-Aligned LLM](https://mlsanity.com/paper/2412.15156/) (2024)

## Built on by (most cited first, 30 of 30)

- [Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation](https://mlsanity.com/paper/2512.04678/) (2025)
- [VISTA: A Test-Time Self-Improving Video Generation Agent](https://mlsanity.com/paper/2510.15831/) (2025)
- [Epipolar Geometry Improves Video Generation Models](https://mlsanity.com/paper/2510.21615/) (2025)
- [Video Generation Models Are Good Latent Reward Models](https://mlsanity.com/paper/2511.21541/) (2025)
- [Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances](https://mlsanity.com/paper/2508.10316/) (2025)
- [VQQA: An Agentic Approach for Video Evaluation and Quality Improvement](https://mlsanity.com/paper/2603.12310/) (2026)
- [Causally Steered Diffusion for Automated Video Counterfactual Generation](https://mlsanity.com/paper/2506.14404/) (2025)
- [RealDPO: Real or Not Real, that is the Preference](https://mlsanity.com/paper/2510.14955/) (2025)
- [CoAgent: Collaborative Planning and Consistency Agent for Coherent Video Generation](https://mlsanity.com/paper/2512.22536/) (2025)
- [BrandFusion: A Multi-Agent Framework for Seamless Brand Integration in Text-to-Video Generation](https://mlsanity.com/paper/2603.02816/) (2026)
- [Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Control](https://mlsanity.com/paper/2606.04775/) (2026)
- [CAPE-T2V: Captioner-Anchored Prompt Enhancement toward Two-Sided Conditioning Alignment in Text-to-Video Generation](https://mlsanity.com/paper/2608.03046/) (2026)
- [Steering Video Diffusion Transformers with Massive Activations](https://mlsanity.com/paper/2603.17825/) (2026)
- [V.I.P. : Iterative Online Preference Distillation for Efficient Video Diffusion Models](https://mlsanity.com/paper/2508.03254/) (2025)
- [Anchored Video Generation: Decoupling Scene Construction and Temporal Synthesis in Text-to-Video Diffusion Models](https://mlsanity.com/paper/2512.16371/) (2025)
- [Bridging Creative Intent and Visual Quality: Creator-Driven Recurrent Video Generation with Agentic Feedback Loops](https://mlsanity.com/paper/2606.18591/) (2026)
- [Flux-OPD: On-Policy Distillation with Evolving Contexts](https://mlsanity.com/paper/2607.28022/) (2026)
- [Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence](https://mlsanity.com/paper/2608.12290/) (2026)
- [HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models](https://mlsanity.com/paper/2608.13205/) (2026)
- [WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generation](https://mlsanity.com/paper/2609.30221/) (2026)
- [A Reason-then-Describe Instruction Interpreter for Controllable Video Generation](https://mlsanity.com/paper/2511.20563/) (2025)
- [P-Flow: Prompting Visual Effects Generation](https://mlsanity.com/paper/2603.22091/) (2026)
- [Comparison Drives Preference: Reference-Aware Modeling for AI-Generated Video Quality Assessment](https://mlsanity.com/paper/2604.17074/) (2026)
- [From Evaluation to Enhancement: Benchmarking and Improving Think-with-Video Reasoning for Video Generative Models](https://mlsanity.com/paper/2609.11242/) (2026)
- [SkillPE: Creativity-Oriented Cinematic Skill Evolution for Text-to-Video Prompt Engineering](https://mlsanity.com/paper/2609.34335/) (2026)
- [DiffWAM: A Fast and Efficient Navigation World Action Model](https://mlsanity.com/paper/2609.39763/) (2026)
- [Towards Unified Evaluation of Prompt Enhancers for Video Generation](https://mlsanity.com/paper/2610.11736/) (2026)
- [Golden Anchor for Video Generation](https://mlsanity.com/paper/2610.13217/) (2026)
- [TIDE: Trajectory-Informed Skill Distillation for Efficient Text-to-Video Prompt Optimization](https://mlsanity.com/paper/2610.15098/) (2026)
- [FlowCred: Hierarchical Credit Assignment and Functional Coordination for Joint Text-to-Video Post-Training](https://mlsanity.com/paper/2610.18142/) (2026)

## Abstract (arXiv)

Video generation models have achieved remarkable progress in text-to-video tasks. These models are typically trained on text-video pairs with highly detailed and carefully crafted descriptions, while real-world user inputs during inference are often concise, vague, or poorly structured. This gap makes prompt optimization crucial for generating high-quality videos. Current methods often rely on large language models (LLMs) to refine prompts through in-context learning, but suffer from several limitations: they may distort user intent, omit critical details, or introduce safety risks. Moreover, they optimize prompts without considering the impact on the final video quality, which can lead to suboptimal results. To address these issues, we introduce VPO, a principled framework that optimizes prompts based on three core principles: harmlessness, accuracy, and helpfulness. The generated prompts faithfully preserve user intents and, more importantly, enhance the safety and quality of generated videos. To achieve this, VPO employs a two-stage optimization approach. First, we construct and refine a supervised fine-tuning (SFT) dataset based on principles of safety and alignment. Second, we introduce both text-level and video-level feedback to further optimize the SFT model with preference learning. Our extensive experiments demonstrate that VPO significantly improves safety, alignment, and video quality compared to baseline methods. Moreover, VPO shows strong generalization across video generation models. Furthermore, we demonstrate that VPO could outperform and be combined with RLHF methods on video generation models, underscoring the effectiveness of VPO in aligning video generation models. Our code and data are publicly available at https://github.com/thu-coai/VPO.