mlsanityData to 14 Oct 2026

CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation

arXiv 2606.09639 ↗First posted 8 Jun 2026cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

The fidelity and structural diversity of training datasets fundamentally determine the capabilities of video generation models. While commercial systems showremarkableabilitytogeneratecinematicnarratives, the progress of open-source models remains limited by the scarcity of high-quality training data. To bridge this gap, we introduce CineDance-1M, a large-scale, open research Text-to-Audio-Video (T2AV) dataset designed specifically for multi-shot, long-form joint audio-video generation. Averaging 92.8 seconds and 24.2 continuous shots per video, it provides configurable, structured annotations for both audio and video modalities. This exceptional quality is achieved through a rigorous three-stage curation pipeline: i) diverse sourcing and comprehensive cleansing, ii) film-theory-inspired narrative parsing, and iii) hierarchical dual-modal captioning. For a comprehensive assessment, we propose CineBench, featuring a diverse prompt suite and a six-dimensional, human-aligned metric system tailored for complex narrative audio-video evaluation. Furthermore, we adapt LTX-2.3 into CineDance, which demonstrates exceptional single-modality quality alongside precise audio-video alignment and robust subject and environment consistency, effectively validating our curation strategy and the high quality of CineDance-1M. We anticipate that this work will serve as a solid foundation for accelerating future research in multi-shot, long-form joint audio-video generation. Our project page is available at https://aliothchen.github.io/projects/CineDance/.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 93 of its 101 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

93 of 101 references.

References · 93

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  5. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  6. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  7. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  8. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  9. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  10. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  11. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  12. 2017VoxCeleb: a large-scale speaker identification datasetcited by 732
38 more
  1. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
  2. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 648
  3. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  4. 2025Qwen3-Omni Technical Reportcited by 497
  5. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
  6. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  7. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  8. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
  9. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  10. 2018LRS3-TED: a large-scale dataset for visual speech recognitioncited by 308
  11. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  12. 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 241
  13. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
  14. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 229
  15. 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
  16. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
  17. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
  18. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 180
  19. 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 171
  20. 2025HunyuanVideo 1.5 Technical Reportcited by 160
  21. 2026Seedance 2.0: Advancing Video Generation for World Complexitycited by 160
  22. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 146
  23. 2025Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Soundcited by 137
  24. 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 135
  25. 2024MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesiscited by 134
  26. 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 131
  27. 2023FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generationcited by 122
  28. 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 89
  29. 2025Automated Movie Generation via Multi-Agent CoT Planningcited by 89
  30. 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 88
  31. 2025HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generationcited by 87
  32. 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 83
  33. 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
  34. 2024ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generationcited by 75
  35. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  36. 2024Evaluation of Text-to-Video Generation Models: A Dynamics Perspectivecited by 67
  37. 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
  38. 2024VidGen-1M: A Large-Scale Dataset for Text-to-video Generationcited by 62

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.