mlsanityData to 7 Oct 2026

MuSS: A Large-Scale Dataset and Cinematic Narrative Benchmark for Multi-Shot Subject-to-Video Generation

arXiv 2604.23789 ↗First posted 26 Apr 2026cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

While video foundation models excel at single-shot generation, real-world cinematic storytelling inherently relies on complex multi-shot sequencing. Further progress is constrained by the absence of datasets that address three core challenges: authentic narrative logic, spatiotemporal text-video alignment conflicts, and the "copy-paste" dilemma prevalent in Subject-to-Video (S2V) generation. To bridge this gap, we introduce MuSS, a large-scale, dual-track dataset tailored for multi-shot video and S2V generation. Sourced from over 3,000 movies, MuSS explicitly supports both complex montage transitions and subject-centric narratives. To construct this dataset, we pioneer a progressive captioning pipeline that eliminates contextual conflicts by ensuring local shot-level accuracy before enforcing global narrative coherence. Crucially, we implement a cross-shot matching mechanism to fundamentally eradicate the S2V copy-paste shortcut. Alongside the dataset, we propose the Cinematic Narrative Benchmark, featuring a visual-logic-driven paradigm and a novel Anti-Copy-Paste Variance (ACP-Var) metric to rigorously assess continuous storytelling and 3D structural consistency. Extensive experiments demonstrate that while current baselines struggle with continuous narrative logic or degenerate into trivial 2D sticker generators, our MuSS-augmented model achieves state-of-the-art narrative effectiveness and cross-shot identity preservation.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 38 of its 56 references that are in the map. First posted on arXiv 26 Apr 2026.

Builds on

38 of 56 references.

References · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  8. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  9. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  10. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  11. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  12. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
26 more
  1. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  2. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  3. 2025VACE: All-in-One Video Creation and Editingcited by 467
  4. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  5. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
  6. 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 240
  7. 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 166
  8. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
  9. 2025Long Context Tuning for Video Generationcited by 110
  10. 2025Mixture of Contexts for Long Video Generationcited by 90
  11. 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 85
  12. 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
  13. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  14. 2023Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videoscited by 54
  15. 2025HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narrativescited by 51
  16. 2025ShotAdapter: Text-to-Multi-Shot Video Generation with Diffusion Modelscited by 37
  17. 2025MagicMirror: ID-Preserved Video Generation in Video Diffusion Transformerscited by 32
  18. 2025EchoShot: Multi-Shot Portrait Video Generationcited by 28
  19. 2025CineTrans: Learning to Generate Videos with Cinematic Transitions via Masked Diffusion Modelscited by 26
  20. 2024LVD-2M: A Long-take Video Dataset with Temporally Dense Captionscited by 24
  21. 2025Kaleido: Open-Sourced Multi-Subject Reference Video Generation Modelcited by 16
  22. 2025Cut2Next: Generating Next Shot via In-Context Tuningcited by 15
  23. 2025AnimeShooter: A Multi-Shot Animation Dataset for Reference-Guided Video Generationcited by 8
  24. 2025FairyGen: Storied Cartoon Video from a Single Child-Drawn Charactercited by 4
  25. 2025OpenSubject: Leveraging Video-Derived Identity and Diversity Priors for Subject-driven Image Generation and Manipulationcited by 4
  26. 2024Multimodal Diffusion Transformer with Memory Bank for Scalable Long-Duration Talking Video Generationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.