MuSS: A Large-Scale Dataset and Cinematic Narrative Benchmark for Multi-Shot Subject-to-Video Generation
Not on the map. Why?
Abstract (arXiv)
While video foundation models excel at single-shot generation, real-world cinematic storytelling inherently relies on complex multi-shot sequencing. Further progress is constrained by the absence of datasets that address three core challenges: authentic narrative logic, spatiotemporal text-video alignment conflicts, and the "copy-paste" dilemma prevalent in Subject-to-Video (S2V) generation. To bridge this gap, we introduce MuSS, a large-scale, dual-track dataset tailored for multi-shot video and S2V generation. Sourced from over 3,000 movies, MuSS explicitly supports both complex montage transitions and subject-centric narratives. To construct this dataset, we pioneer a progressive captioning pipeline that eliminates contextual conflicts by ensuring local shot-level accuracy before enforcing global narrative coherence. Crucially, we implement a cross-shot matching mechanism to fundamentally eradicate the S2V copy-paste shortcut. Alongside the dataset, we propose the Cinematic Narrative Benchmark, featuring a visual-logic-driven paradigm and a novel Anti-Copy-Paste Variance (ACP-Var) metric to rigorously assess continuous storytelling and 3D structural consistency. Extensive experiments demonstrate that while current baselines struggle with continuous narrative logic or degenerate into trivial 2D sticker generators, our MuSS-augmented model achieves state-of-the-art narrative effectiveness and cross-shot identity preservation.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 38 of its 56 references that are in the map. First posted on arXiv 26 Apr 2026.
Builds on
38 of 56 references.
References · 38
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
26 more
- 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
- 2025VACE: All-in-One Video Creation and Editingcited by 467
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
- 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
- 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 240
- 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 166
- 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
- 2025Long Context Tuning for Video Generationcited by 110
- 2025Mixture of Contexts for Long Video Generationcited by 90
- 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 85
- 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
- 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
- 2023Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videoscited by 54
- 2025HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narrativescited by 51
- 2025ShotAdapter: Text-to-Multi-Shot Video Generation with Diffusion Modelscited by 37
- 2025MagicMirror: ID-Preserved Video Generation in Video Diffusion Transformerscited by 32
- 2025EchoShot: Multi-Shot Portrait Video Generationcited by 28
- 2025CineTrans: Learning to Generate Videos with Cinematic Transitions via Masked Diffusion Modelscited by 26
- 2024LVD-2M: A Long-take Video Dataset with Temporally Dense Captionscited by 24
- 2025Kaleido: Open-Sourced Multi-Subject Reference Video Generation Modelcited by 16
- 2025Cut2Next: Generating Next Shot via In-Context Tuningcited by 15
- 2025AnimeShooter: A Multi-Shot Animation Dataset for Reference-Guided Video Generationcited by 8
- 2025FairyGen: Storied Cartoon Video from a Single Child-Drawn Charactercited by 4
- 2025OpenSubject: Leveraging Video-Derived Identity and Diversity Priors for Subject-driven Image Generation and Manipulationcited by 4
- 2024Multimodal Diffusion Transformer with Memory Bank for Scalable Long-Duration Talking Video Generationcited by 2
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026Infinite Worlds with Versatile Interactionscited by 37
- 2026EvalVerse: Pipeline-Aware and Expert-Calibrated Benchmarking for Professional Cinematic Video Generationcited by 5
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generationcited by 5
- 2026KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generationcited by 1
- 2026PersonaShot: Benchmarking Person-Centric Narrative Continuity in Multi-Shot Video Generationcited by 0
- 2026DramaChain Bench: An End-to-End Benchmark for Short-Drama Generationcited by 0
- 2026OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generationcited by 0
- 2026AdSpark: A Large-Scale Dataset and Benchmark for Product-Centric Advertisement Video Generationcited by 0
- 2026AdSpark: A Large-Scale Dataset and Benchmark for Product-Centric Advertisement Video Generation7 Oct 2026
- 2026OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation18 Sep 2026
- 2026DramaChain Bench: An End-to-End Benchmark for Short-Drama Generation1 Sep 2026
- 2026PersonaShot: Benchmarking Person-Centric Narrative Continuity in Multi-Shot Video Generation17 Aug 2026
- 2026KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation15 Jul 2026
- 2026Infinite Worlds with Versatile Interactions8 Jul 2026
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation8 Jun 2026
- 2026EvalVerse: Pipeline-Aware and Expert-Calibrated Benchmarking for Professional Cinematic Video Generation22 May 2026