# SmartDirector: Keyframe-Conditioned Cinematic Video Generation with Narrative Pacing Control

arXiv: https://arxiv.org/abs/2605.27891 · HTML: https://mlsanity.com/paper/2605.27891/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 31 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 25 of 25)

- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [LTX-Video: Realtime Video Latent Diffusion](https://mlsanity.com/paper/2501.00103/) (2024)
- [Investigating Tradeoffs in Real-World Video Super-Resolution](https://mlsanity.com/paper/2111.12704/) (2021)
- [Goku: Flow Based Video Generative Foundation Models](https://mlsanity.com/paper/2502.04896/) (2025)
- [SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration](https://mlsanity.com/paper/2501.01320/) (2025)
- [UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captions](https://mlsanity.com/paper/2506.13691/) (2025)
- [HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narratives](https://mlsanity.com/paper/2510.20822/) (2025)
- [Captain Cinema: Towards Short Movie Generation](https://mlsanity.com/paper/2507.18634/) (2025)
- [FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolution](https://mlsanity.com/paper/2510.12747/) (2025)
- [MultiShotMaster: A Controllable Multi-Shot Video Generation Framework](https://mlsanity.com/paper/2512.03041/) (2025)
- [DOVE: Efficient One-Step Diffusion Model for Real-World Video Super-Resolution](https://mlsanity.com/paper/2505.16239/) (2025)
- [AutoShot: A Short Video Dataset and State-of-the-Art Shot Boundary Detection](https://mlsanity.com/paper/2304.06116/) (2023)
- [Learning Temporal Coherence via Self-Supervision for GAN-based Video Generation](https://mlsanity.com/paper/1811.09393/) (2018)
- [DreaMontage: Arbitrary Frame-Guided One-Shot Video Generation](https://mlsanity.com/paper/2512.21252/) (2025)
- [SparkVSR: Interactive Video Super-Resolution via Sparse Keyframe Propagation](https://mlsanity.com/paper/2603.16864/) (2026)

## Built on by (most cited first, 2 of 2)

- [KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation](https://mlsanity.com/paper/2607.14202/) (2026)
- [DramaChain Bench: An End-to-End Benchmark for Short-Drama Generation](https://mlsanity.com/paper/2609.00646/) (2026)

## Abstract (arXiv)

The narrative quality of a video fundamentally determines its perceptual value. Although existing video generation methods can produce visually appealing content, they predominantly rely on sparse conditioning signals such as text prompts or first/last frames, which limits precise control over narrative structure and temporal pacing. In this paper, we propose SmartDirector, a framework that enhances the narrative capacity of video generation models through multiple keyframes. SmartDirector supports flexible generation scenarios including single-shot generation, multi-shot narrative synthesis, and video extension. The framework operates in two stages: Director-Gen generates a low-resolution video conditioned on the provided keyframes, and Director-SR refines the output by exploiting high-resolution keyframes as semantic anchors to recover fine-grained details. To enable robust multi-keyframe training, we construct a data pipeline that curates single-shot and multi-shot sequences from movies. Extensive experiments demonstrate that SmartDirector substantially outperforms existing state-of-the-art approaches. We will release the code to facilitate further research.