CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation
Not on the map. Why?
Abstract (arXiv)
The fidelity and structural diversity of training datasets fundamentally determine the capabilities of video generation models. While commercial systems showremarkableabilitytogeneratecinematicnarratives, the progress of open-source models remains limited by the scarcity of high-quality training data. To bridge this gap, we introduce CineDance-1M, a large-scale, open research Text-to-Audio-Video (T2AV) dataset designed specifically for multi-shot, long-form joint audio-video generation. Averaging 92.8 seconds and 24.2 continuous shots per video, it provides configurable, structured annotations for both audio and video modalities. This exceptional quality is achieved through a rigorous three-stage curation pipeline: i) diverse sourcing and comprehensive cleansing, ii) film-theory-inspired narrative parsing, and iii) hierarchical dual-modal captioning. For a comprehensive assessment, we propose CineBench, featuring a diverse prompt suite and a six-dimensional, human-aligned metric system tailored for complex narrative audio-video evaluation. Furthermore, we adapt LTX-2.3 into CineDance, which demonstrates exceptional single-modality quality alongside precise audio-video alignment and robust subject and environment consistency, effectively validating our curation strategy and the high quality of CineDance-1M. We anticipate that this work will serve as a solid foundation for accelerating future research in multi-shot, long-form joint audio-video generation. Our project page is available at https://aliothchen.github.io/projects/CineDance/.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 93 of its 101 references that are in the map. First posted on arXiv 8 Jun 2026.
Builds on
93 of 101 references.
References · 93
- 2025Qwen3 Technical Reportcited by 8,820
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
- 2017VoxCeleb: a large-scale speaker identification datasetcited by 732
38 more
- 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
- 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 648
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
- 2025Qwen3-Omni Technical Reportcited by 497
- 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
- 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
- 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
- 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
- 2018LRS3-TED: a large-scale dataset for visual speech recognitioncited by 308
- 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
- 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 241
- 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
- 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 229
- 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
- 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
- 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
- 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 180
- 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 171
- 2025HunyuanVideo 1.5 Technical Reportcited by 160
- 2026Seedance 2.0: Advancing Video Generation for World Complexitycited by 160
- 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 146
- 2025Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Soundcited by 137
- 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 135
- 2024MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesiscited by 134
- 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 131
- 2023FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generationcited by 122
- 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 89
- 2025Automated Movie Generation via Multi-Agent CoT Planningcited by 89
- 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 88
- 2025HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generationcited by 87
- 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 83
- 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
- 2024ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generationcited by 75
- 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
- 2024Evaluation of Text-to-Video Generation Models: A Dynamics Perspectivecited by 67
- 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
- 2024VidGen-1M: A Large-Scale Dataset for Text-to-video Generationcited by 62
50 most cited of 93 All 93 (JSON)
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Vorch-Director: Interactive World Story Model via Noise-Aware Error Rectificationcited by 1
- 2026Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attentioncited by 1
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Soundcited by 0
- 2026PersonaShot: Benchmarking Person-Centric Narrative Continuity in Multi-Shot Video Generationcited by 0
- 2026DramaChain Bench: An End-to-End Benchmark for Short-Drama Generationcited by 0
- 2026Structured Stateful Relational Scripts with a High-Quality Dataset for Multi-Shot Audio-Visual Video Generationcited by 0
- 2026Structured Stateful Relational Scripts with a High-Quality Dataset for Multi-Shot Audio-Visual Video Generation11 Oct 2026
- 2026DramaChain Bench: An End-to-End Benchmark for Short-Drama Generation1 Sep 2026
- 2026PersonaShot: Benchmarking Person-Centric Narrative Continuity in Multi-Shot Video Generation17 Aug 2026
- 2026Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attention16 Aug 2026
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Sound6 Aug 2026
- 2026Vorch-Director: Interactive World Story Model via Noise-Aware Error Rectification6 Aug 2026