mlsanityData to 7 Oct 2026

MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captions

arXiv 2407.06358 ↗First posted 8 Jul 2024cs.CVcited by 144

Not on the map. Why?

Cited by 35 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Sora's high-motion intensity and long consistent videos have significantly impacted the field of video generation, attracting unprecedented attention. However, existing publicly available datasets are inadequate for generating Sora-like videos, as they mainly contain short videos with low motion intensity and brief captions. To address these issues, we propose MiraData, a high-quality video dataset that surpasses previous ones in video duration, caption detail, motion strength, and visual quality. We curate MiraData from diverse, manually selected sources and meticulously process the data to obtain semantically consistent clips. GPT-4V is employed to annotate structured captions, providing detailed descriptions from four different perspectives along with a summarized dense caption. To better assess temporal consistency and motion intensity in video generation, we introduce MiraBench, which enhances existing benchmarks by adding 3D consistency and tracking-based motion strength metrics. MiraBench includes 150 evaluation prompts and 17 metrics covering temporal consistency, motion strength, 3D consistency, visual quality, text-video alignment, and distribution similarity. To demonstrate the utility and effectiveness of MiraData, we conduct experiments using our DiT-based video generation model, MiraDiT. The experimental results on MiraBench demonstrate the superiority of MiraData, especially in motion strength.

Key facts. Cited by 144 papers in the map (data through 7 Oct 2026). Builds on 65 of its 74 references that are in the map. First posted on arXiv 8 Jul 2024.

Builds on

65 of 74 references.

References · 65

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2022Scalable Diffusion Models with Transformerscited by 5,931
  10. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
38 more
  1. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  4. 2022Video Diffusion Modelscited by 2,280
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  6. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  7. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  8. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  9. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  10. 2018Demystifying MMD GANscited by 1,268
  11. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  12. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  13. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  14. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  15. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  16. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  17. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  18. 2017Localizing Moments in Video with Natural Languagecited by 660
  19. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
  20. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  21. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  22. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  23. 2023CoTracker: It is Better to Track Togethercited by 516
  24. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  25. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  26. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  27. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  28. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  29. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 412
  30. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  31. 2016Generating Videos with Scene Dynamicscited by 376
  32. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
  33. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  34. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  35. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  36. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  37. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 278
  38. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 75; 2026: 45.

144 papers.

Sort:

By area:

  1. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
  2. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
  3. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
  4. 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 132
  5. 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 127
  6. 2024GameGen-X: Interactive Open-world Game Video Generationcited by 124
  7. 2025Video World Models with Long-term Spatial Memorycited by 124
  8. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  9. 2025Controllable Video Generation: A Surveycited by 93
  10. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Modelcited by 73
  11. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 72
  12. 2025Goku: Flow Based Video Generative Foundation Modelscited by 67
38 more
  1. 2024Progressive Autoregressive Video Diffusion Modelscited by 65
  2. 2025Let Them Talk: Audio-Driven Multi-Person Conversational Video Generationcited by 64
  3. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
  4. 2024VidGen-1M: A Large-Scale Dataset for Text-to-video Generationcited by 60
  5. 2025MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidancecited by 60
  6. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 59
  7. 2025Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Modelscited by 58
  8. 2024T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Designcited by 54
  9. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
  10. 2025OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLMcited by 53
  11. 2025UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captionscited by 52
  12. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
  13. 2024Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedbackcited by 45
  14. 2025FullDiT: Multi-Task Video Generative Foundation Model with Full Attentioncited by 45
  15. 2025InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generationcited by 43
  16. 2024ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generationcited by 41
  17. 2024LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexitycited by 41
  18. 2024Mind the Time: Temporally-Controlled Multi-Event Video Generationcited by 39
  19. 2025Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmarkcited by 37
  20. 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 35
  21. 2024xGen-MM-Vid (BLIP-3-Video): You Only Need 32 Tokens to Represent a Video Even in VLMscited by 34
  22. 2025Mavors: Multi-granularity Video Representation for Multimodal Large Language Modelcited by 31
  23. 2025CineTrans: Learning to Generate Videos with Cinematic Transitions via Masked Diffusion Modelscited by 26
  24. 2025Pack and Force Your Memory: Long-form and Consistent Video Generationcited by 26
  25. 2026VINO: A Unified Visual Generator with Interleaved OmniModal Contextcited by 26
  26. 2024PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Modelscited by 25
  27. 2025FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarioscited by 25
  28. 2025VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generationcited by 22
  29. 2025Epipolar Geometry Improves Video Generation Modelscited by 22
  30. 2025Generative Video Motion Editing with 3D Point Trackscited by 21
  31. 2025VideoUFO: A Million-Scale User-Focused Dataset for Text-to-Video Generationcited by 20
  32. 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 20
  33. 2025Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generationcited by 20
  34. 2025The Quest for Generalizable Motion Generation: Data, Model, and Evaluationcited by 20
  35. 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 20
  36. 2026VGGRPO: Towards World-Consistent Video Generation with 4D Latent Rewardcited by 19
  37. 2024InstanceCap: Improving Text-to-Video Generation via Instance-aware Structured Captioncited by 17
  38. 2024Owl-1: Omni World Model for Consistent Long Video Generationcited by 17

50 most cited of 144 All 144 (JSON)