# CAPE-T2V: Captioner-Anchored Prompt Enhancement toward Two-Sided Conditioning Alignment in Text-to-Video Generation

arXiv: https://arxiv.org/abs/2608.03046 · HTML: https://mlsanity.com/paper/2608.03046/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 21 of 21)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Movie Gen: A Cast of Media Foundation Models](https://mlsanity.com/paper/2410.13720/) (2024)
- [OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation](https://mlsanity.com/paper/2407.02371/) (2024)
- [VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness](https://mlsanity.com/paper/2503.21755/) (2025)
- [Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking](https://mlsanity.com/paper/2601.04720/) (2026)
- [T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation](https://mlsanity.com/paper/2407.14505/) (2024)
- [Movie Description](https://mlsanity.com/paper/1605.03705/) (2016)
- [MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captions](https://mlsanity.com/paper/2407.06358/) (2024)
- [Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content](https://mlsanity.com/paper/2410.08260/) (2024)
- [VideoDPO: Omni-Preference Alignment for Video Diffusion Generation](https://mlsanity.com/paper/2412.14167/) (2024)
- [VPO: Aligning Text-to-Video Generation Models with Prompt Optimization](https://mlsanity.com/paper/2503.20491/) (2025)
- [Prompt-A-Video: Prompt Your Video Diffusion Model via Preference-Aligned LLM](https://mlsanity.com/paper/2412.15156/) (2024)
- [VideoUFO: A Million-Scale User-Focused Dataset for Text-to-Video Generation](https://mlsanity.com/paper/2503.01739/) (2025)
- [The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generation](https://mlsanity.com/paper/2504.11739/) (2025)
- [InstanceCap: Improving Text-to-Video Generation via Instance-aware Structured Caption](https://mlsanity.com/paper/2412.09283/) (2024)
- [StoryTeller: Improving Long Video Description through Global Audio-Visual Character Identification](https://mlsanity.com/paper/2411.07076/) (2024)
- [PromptRL: Prompt Matters in RL for Flow-Based Image Generation](https://mlsanity.com/paper/2602.01382/) (2026)
- [RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via Data Alignment and Test-Time Scaling](https://mlsanity.com/paper/2510.20206/) (2025)
- [VC4VG: Optimizing Video Captions for Text-to-Video Generation](https://mlsanity.com/paper/2510.24134/) (2025)
- [POS: A Prompts Optimization Suite for Augmenting Text-to-Video Generation](https://mlsanity.com/paper/2311.00949/) (2023)

## Built on by (most cited first, 3 of 3)

- [WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generation](https://mlsanity.com/paper/2609.30221/) (2026)
- [Towards Unified Evaluation of Prompt Enhancers for Video Generation](https://mlsanity.com/paper/2610.11736/) (2026)
- [FlowCred: Hierarchical Credit Assignment and Functional Coordination for Joint Text-to-Video Post-Training](https://mlsanity.com/paper/2610.18142/) (2026)

## Abstract (arXiv)

Text-to-video (T2V) diffusion transformers (DiTs) are trained with detailed video captions, whereas inference often relies on user prompts rewritten by a prompt enhancer (PE). Prior work has improved generation by optimizing the PE, the DiT, or both; some methods have also sought to narrow the training-inference mismatch through shared schemas. Yet even within a shared schema, inference-time PE outputs and DiT training captions may still differ in detail selection, information organization, descriptive granularity, and phrasing. We refer to this residual mismatch as the PE-Caption gap and introduce CAPE-T2V, a two-step Captioner-Anchored Prompt Enhancement framework toward two-sided conditioning alignment in T2V generation. First, CAPE-T2V constructs three types of PE training examples, pairing captioner-generated targets with concise source captions, detailed source captions, or pseudo user prompts derived from those targets. It then fine-tunes the PE to map each input to its paired target. Second, CAPE-T2V fine-tunes the DiT on video-derived captions rewritten by the Anchored PE; the same PE rewrites user prompts at inference. Relative to a baseline using the same caption schema, CAPE-T2V achieves higher aggregate scores on StoryEval, VBench-2.0, and T2V-CompBench across Wan2.2 and LTX-2.3. Further, CAPE-T2V exhibits a smaller PE-Caption gap than the baseline: its DiT fine-tuning captions are closer in distribution to inference-time PE outputs, as measured by squared maximum mean discrepancy in a fixed embedding space. Overall, these results support CAPE-T2V as an effective approach to mitigating the PE-Caption gap. The project is available at https://github.com/yizzz927/CAPE-T2V.