RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via Data Alignment and Test-Time Scaling
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Prompt design plays a crucial role in text-to-video (T2V) generation, yet user-provided prompts are often short, unstructured, and misaligned with training data, limiting the generative potential of diffusion-based T2V models. We present RAPO++, a cross-stage prompt optimization framework that unifies training-data--aligned refinement, test-time iterative scaling, and large language model (LLM) fine-tuning to substantially improve T2V generation without modifying the underlying generative backbone. In Stage 1, Retrieval-Augmented Prompt Optimization (RAPO) enriches user prompts with semantically relevant modifiers retrieved from a relation graph and refactors them to match training distributions, enhancing compositionality and multi-object fidelity. Stage 2 introduces Sample-Specific Prompt Optimization (SSPO), a closed-loop mechanism that iteratively refines prompts using multi-source feedback -- including semantic alignment, spatial fidelity, temporal coherence, and task-specific signals such as optical flow -- yielding progressively improved video generation quality. Stage 3 leverages optimized prompt pairs from SSPO to fine-tune the rewriter LLM, internalizing task-specific optimization patterns and enabling efficient, high-quality prompt generation even before inference. Extensive experiments across five state-of-the-art T2V models and five benchmarks demonstrate that RAPO++ achieves significant gains in semantic alignment, compositional reasoning, temporal stability, and physical plausibility, outperforming existing methods by large margins. Our results highlight RAPO++ as a model-agnostic, cost-efficient, and scalable solution that sets a new standard for prompt optimization in T2V generation. The code is available at https://github.com/Vchitect/RAPO.
Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 88 of its 94 references that are in the map. First posted on arXiv 23 Oct 2025.
Builds on
88 of 94 references.
Closest ancestors · 2
Same direction, newest first
- 2025PromptEnhancer: A Simple Approach to Enhance Text-to-Image Models via Chain-of-Thought Prompt Rewriting4 Sep
- 2025RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning23 May
Other references · 86
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023GPT-4 Technical Reportcited by 18,396
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
38 more
- 2023Qwen Technical Reportcited by 3,344
- 2023Mistral 7Bcited by 3,164
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2022Video Diffusion Modelscited by 2,305
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
- 2025s1: Simple test-time scalingcited by 1,341
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
- 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 652
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
- 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
- 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 579
- 2024Latte: Latent Diffusion Transformer for Video Generationcited by 546
- 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
- 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388
- 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
- 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
- 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 307
- 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 302
- 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 234
- 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 233
- 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
- 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
- 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 218
- 2022Optimizing Prompts for Text-to-Image Generationcited by 207
- 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
- 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 189
- 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
- 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 180
50 most cited of 86 All 86 (JSON)
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026Paper2Rebuttal: A Multi-Agent Framework for Transparent Author Response Assistancecited by 9
- 2026Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasonerscited by 3
- 2026CAPE-T2V: Captioner-Anchored Prompt Enhancement toward Two-Sided Conditioning Alignment in Text-to-Video Generationcited by 3
- 2026Guiding Diffusion-based Reconstruction with Contrastive Signals for Balanced Visual Representationcited by 2
- 2026When Cultures Move: Measuring and Improving Multicultural Text-to-Video Generationcited by 1
- 2026WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generationcited by 1
- 2026Test-Time Scaling in Multimodal Foundation Models: A Comprehensive Survey of Generation and Reasoningcited by 0
- 2026Test-Time Scaling for Video Diffusion Models via Diagnosis-Guided Candidate Recyclingcited by 0
- 2026Towards Unified Evaluation of Prompt Enhancers for Video Generationcited by 0
- 2026TIDE: Trajectory-Informed Skill Distillation for Efficient Text-to-Video Prompt Optimizationcited by 0
- 2026TIDE: Trajectory-Informed Skill Distillation for Efficient Text-to-Video Prompt Optimization10 Oct 2026
- 2026Towards Unified Evaluation of Prompt Enhancers for Video Generation8 Oct 2026
- 2026WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generation24 Sep 2026
- 2026Test-Time Scaling for Video Diffusion Models via Diagnosis-Guided Candidate Recycling29 Aug 2026
- 2026CAPE-T2V: Captioner-Anchored Prompt Enhancement toward Two-Sided Conditioning Alignment in Text-to-Video Generation4 Aug 2026
- 2026Test-Time Scaling in Multimodal Foundation Models: A Comprehensive Survey of Generation and Reasoning6 Jun 2026
- 2026When Cultures Move: Measuring and Improving Multicultural Text-to-Video Generation16 May 2026
- 2026Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasoners14 May 2026
- 2026Guiding Diffusion-based Reconstruction with Contrastive Signals for Balanced Visual Representation5 Mar 2026
- 2026Paper2Rebuttal: A Multi-Agent Framework for Transparent Author Response Assistance20 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025VPO: Aligning Text-to-Video Generation Models with Prompt Optimization
- 2024Prompt-A-Video: Prompt Your Video Diffusion Model via Preference-Aligned LLM
- 2023POS: A Prompts Optimization Suite for Augmenting Text-to-Video Generation
- 2026Retrieval, Refinement, and Ranking for Text-to-Video Generation via Prompt Optimization and Test-Time Scaling
- 2025ConceptMix++: Leveling the Playing Field in Text-to-Image Benchmarking via Iterative Prompt Optimization
- 2025Multimodal Prompt Optimization: Why Not Leverage Multiple Modalities for MLLMs
- 2024Batch-Instructed Gradient for Prompt Evolution:Systematic Prompt Optimization for Enhanced Text-to-Image Synthesis
- 2025POET: Preference Optimization for Enhanced Text-to-Image Generation