# A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation

arXiv: https://arxiv.org/abs/2604.10456 · HTML: https://mlsanity.com/paper/2604.10456/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 51 references that are in the map.
- First posted on arXiv 12 Apr 2026.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [Chain of Agents: Large Language Models Collaborating on Long-Context Tasks](https://mlsanity.com/paper/2406.02818/) (2024)
- [Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit](https://mlsanity.com/paper/2210.17016/) (2022)
- [Beyond Appearance: a Semantic Controllable Self-Supervised Learning Framework for Human-Centric Visual Tasks](https://mlsanity.com/paper/2303.17602/) (2023)
- [Automated Movie Generation via Multi-Agent CoT Planning](https://mlsanity.com/paper/2503.07314/) (2025)
- [LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing](https://mlsanity.com/paper/2402.10294/) (2024)
- [Learning Trailer Moments in Full-Length Movies](https://mlsanity.com/paper/2008.08502/) (2020)
- [AutoShot: A Short Video Dataset and State-of-the-Art Shot Boundary Detection](https://mlsanity.com/paper/2304.06116/) (2023)
- [STAGE: Storyboard-Anchored Generation for Cinematic Multi-shot Narrative](https://mlsanity.com/paper/2512.12372/) (2025)
- [Rescribe: Authoring and Automatically Editing Audio Descriptions](https://mlsanity.com/paper/2010.03667/) (2020)
- [AVscript: Accessible Video Editing with Audio-Visual Scripts](https://mlsanity.com/paper/2302.14117/) (2023)
- [Learning to Cut by Watching Movies](https://mlsanity.com/paper/2108.04294/) (2021)
- [Boundary-aware Self-supervised Learning for Video Scene Segmentation](https://mlsanity.com/paper/2201.05277/) (2022)
- [B-Script: Transcript-based B-roll Video Editing with Recommendations](https://mlsanity.com/paper/1902.11216/) (2019)
- [EditDuet: A Multi-Agent System for Video Non-Linear Editing](https://mlsanity.com/paper/2509.10761/) (2025)
- [Transcript to Video: Efficient Clip Sequencing from Texts](https://mlsanity.com/paper/2107.11851/) (2021)
- [Towards Automated Movie Trailer Generation](https://mlsanity.com/paper/2404.03477/) (2024)
- [A Reinforcement Learning-Based Automatic Video Editing Method Using Pre-trained Vision-Language Model](https://mlsanity.com/paper/2411.04942/) (2024)

## Built on by (most cited first, 6 of 6)

- [MolMem: Memory-Augmented Agentic Reinforcement Learning for Sample-Efficient Molecular Optimization](https://mlsanity.com/paper/2604.12237/) (2026)
- [BEAT: Rhythm-Elastic Alignment for Agentic Music-guided Movie Trailer Generation](https://mlsanity.com/paper/2605.27067/) (2026)
- [Crayotter: Traceable Multi-Agent Workflows for Long-Form Video Editing](https://mlsanity.com/paper/2606.07636/) (2026)
- [Crayotter: Learning Long-Horizon Video Editing Agents via Group-Relative Preference Backpropagation](https://mlsanity.com/paper/2608.02694/) (2026)
- [Agentic Visual Generation: From Generative Models to Agentic Control](https://mlsanity.com/paper/2609.06758/) (2026)
- [FRAMEWORKERS: A Dynamic Multi-Agent Framework for AI-Generated Video Production](https://mlsanity.com/paper/2608.29814/) (2026)

## Abstract (arXiv)

The surging demand for adapting long-form cinematic content into short videos has motivated the need for versatile automatic video compilation systems. However, existing compilation methods are limited to predefined tasks, and the community lacks a comprehensive benchmark to evaluate the cinematic compilation. To address this, we introduce CineBench, the first benchmark for instruction-driven cinematic video compilation, featuring diverse user instructions and high-quality ground-truth compilations annotated by professional editors. To overcome contextual collapse and temporal fragmentation, we present CineAgents, a multi-agent system that reformulates cinematic video compilation into ``design-and-compose'' paradigm. CineAgents performs script reverse-engineering to construct a hierarchical narrative memory to provide multi-level context and employs an iterative narrative planning process that refines a creative blueprint into a final compiled script. Extensive experiments demonstrate that CineAgents significantly outperforms existing methods, generating compilations with superior narrative coherence and logical coherence.