mlsanityData to 14 Oct 2026

Open-Sora Plan: Open-Source Large Video Generation Model

arXiv 2412.00131 ↗First posted 28 Nov 2024cs.CV cs.AIcited by 296

Not on the map. Why?

Cited by 114 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Open-Sora Plan, an open-source project that aims to contribute a large generation model for generating desired high-resolution videos with long durations based on various user inputs. Our project comprises multiple components for the entire video generation process, including a Wavelet-Flow Variational Autoencoder, a Joint Image-Video Skiparse Denoiser, and various condition controllers. Moreover, many assistant strategies for efficient training and inference are designed, and a multi-dimensional data curation pipeline is proposed for obtaining desired high-quality data. Benefiting from efficient thoughts, our Open-Sora Plan achieves impressive video generation results in both qualitative and quantitative evaluations. We hope our careful design and practical experience can inspire the video generation research community. All our codes and model weights are publicly available at urlhttps://github.com/PKU-YuanGroup/Open-Sora-Plan.

Key facts. Cited by 296 papers in the map (data through 14 Oct 2026). Builds on 72 of its 81 references that are in the map. First posted on arXiv 28 Nov 2024.

Builds on

72 of 81 references.

References · 72

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2024The Llama 3 Herd of Modelscited by 14,715
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  8. 2020Denoising Diffusion Implicit Modelscited by 9,781
  9. 2023Segment Anythingcited by 8,234
  10. 2023Visual Instruction Tuningcited by 7,869
  11. 2022Scalable Diffusion Models with Transformerscited by 6,054
  12. 2022Flow Matching for Generative Modelingcited by 5,702
38 more
  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  8. 2023Mistral 7Bcited by 3,164
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  10. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  11. 2024Qwen2 Technical Reportcited by 2,120
  12. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  13. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  14. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  15. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  16. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  17. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  18. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  19. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  20. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  21. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  22. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  23. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  24. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  25. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  26. 2024Yi: Open Foundation Models by 01.AIcited by 719
  27. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  28. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  29. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  30. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
  31. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
  32. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
  33. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 379
  34. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
  35. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 370
  36. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 322
  37. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
  38. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 310

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 166; 2026: 119.

296 papers.

Sort:

By area:

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  2. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  3. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 652
  4. 2025MAGI-1: Autoregressive Video Generation at Scalecited by 276
  5. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  6. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 231
  7. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
  8. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 192
  9. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 186
  10. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 185
  11. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 184
  12. 2025History-Guided Video Diffusioncited by 171
38 more
  1. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 168
  2. 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 163
  3. 2025Video World Models with Long-term Spatial Memorycited by 136
  4. 2025Fast Video Generation with Sliding Tile Attentioncited by 128
  5. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 125
  6. 2025SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inferencecited by 120
  7. 2025VSA: Faster Video Diffusion with Trainable Sparse Attentioncited by 104
  8. 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 98
  9. 2025Controllable Video Generation: A Surveycited by 98
  10. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 94
  11. 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 86
  12. 2025Seaweed-7B: Cost-Effective Training of Video Generation Foundation Modelcited by 82
  13. 2025Improving the Diffusability of Autoencoderscited by 80
  14. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  15. 2025Stable Video Infinity: Infinite-Length Video Generation with Error Recyclingcited by 73
  16. 2025Training-free and Adaptive Sparse Attention for Efficient Long Video Generationcited by 69
  17. 2025Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generationcited by 69
  18. 2025RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformerscited by 66
  19. 2025UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generationcited by 66
  20. 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
  21. 2025GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generationcited by 58
  22. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 57
  23. 2025UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captionscited by 57
  24. 2026Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generationcited by 56
  25. 2025OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editingcited by 53
  26. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 49
  27. 2025Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systemscited by 48
  28. 2025FullDiT: Multi-Task Video Generative Foundation Model with Full Attentioncited by 47
  29. 2025Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidancecited by 45
  30. 2025Pre-Trained Video Generative Models as World Simulatorscited by 44
  31. 2025ATI: Any Trajectory Instruction for Controllable Video Generationcited by 43
  32. 2025VideoScore2: Think before You Score in Generative Video Evaluationcited by 40
  33. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 40
  34. 2025"PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Modelscited by 36
  35. 2025MultiShotMaster: A Controllable Multi-Shot Video Generation Frameworkcited by 36
  36. 2025MagicMirror: ID-Preserved Video Generation in Video Diffusion Transformerscited by 34
  37. 2025Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Controlcited by 34
  38. 2025Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformerscited by 34

50 most cited of 296 All 296 (JSON)