# SkillPE: Creativity-Oriented Cinematic Skill Evolution for Text-to-Video Prompt Engineering

arXiv: https://arxiv.org/abs/2609.34335 · HTML: https://mlsanity.com/paper/2609.34335/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills](https://mlsanity.com/paper/2504.07079/) (2025)
- [Movie Description](https://mlsanity.com/paper/1605.03705/) (2016)
- [Seedance 2.0: Advancing Video Generation for World Complexity](https://mlsanity.com/paper/2604.14148/) (2026)
- [Reinforcement Learning for Self-Improving Agent with Skill Library](https://mlsanity.com/paper/2512.17102/) (2025)
- [Automated Movie Generation via Multi-Agent CoT Planning](https://mlsanity.com/paper/2503.07314/) (2025)
- [T2I-Copilot: A Training-Free Multi-Agent Text-to-Image System for Enhanced Prompt Interpretation and Interactive Generation](https://mlsanity.com/paper/2507.20536/) (2025)
- [VISTA: A Test-Time Self-Improving Video Generation Agent](https://mlsanity.com/paper/2510.15831/) (2025)
- [Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation](https://mlsanity.com/paper/2412.16211/) (2024)
- [VPO: Aligning Text-to-Video Generation Models with Prompt Optimization](https://mlsanity.com/paper/2503.20491/) (2025)
- [Movie101: A New Movie Understanding Benchmark](https://mlsanity.com/paper/2305.12140/) (2023)
- [GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning](https://mlsanity.com/paper/2601.18543/) (2026)
- [Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generation](https://mlsanity.com/paper/2602.01756/) (2026)
- [The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generation](https://mlsanity.com/paper/2504.11739/) (2025)
- [MAViS: A Multi-Agent Framework for Long-Sequence Video Storytelling](https://mlsanity.com/paper/2508.08487/) (2025)
- [GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation](https://mlsanity.com/paper/2605.21605/) (2026)
- [VQQA: An Agentic Approach for Video Evaluation and Quality Improvement](https://mlsanity.com/paper/2603.12310/) (2026)
- [NEWTON: Agentic Planning for Physically Grounded Video Generation](https://mlsanity.com/paper/2605.18396/) (2026)
- [V-Stylist: Video Stylization via Collaboration and Reflection of MLLM Agents](https://mlsanity.com/paper/2503.12077/) (2025)
- [VideoAgent: All-in-One Framework for Video Understanding and Editing](https://mlsanity.com/paper/2606.23327/) (2026)

## Abstract (arXiv)

Achieving high-quality, cinematic results in text-to-video generation remains challenging for non-experts, whose prompts often lack professional narrative and creative design. We propose SkillPE, a prompt engineering (PE) framework that evolves reusable cinematic skills from expert-authored seeds. SkillPE represents shot logic, composition, lighting, sound design, and other filmmaking cues in a fine-grained format, and retrieves movie references categorized as resonators (good matches), dissonants (weak matches), and divergents (creatively useful near-misses). The first two refine when and how a skill should be applied, while divergents inspire alternative cinematic realizations at different degrees of modification while preserving the user intent. Candidate skills are assessed through generated videos along prompt fidelity, cinematic quality, narrative appeal, and creativity to construct the final skill libraries. Experiments on StoryEval and VBench show improvements of up to 1.40 points over the strongest external baseline and 0.51 points over seed skills on 7-point four-dimensional evaluation, while remaining competitive on benchmark-native metrics. Overall, SkillPE offers a practical approach to balancing fidelity and creativity in cinematic text-to-video generation. Code is available at https://github.com/Ais0n/SkillPE .