mlsanityData to 7 Oct 2026

Open-Sora: Democratizing Efficient Video Production for All

arXiv 2412.20404 ↗First posted 29 Dec 2024cs.CVcited by 637

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Vision and language are the two foundational senses for humans, and they build up our cognitive ability and intelligence. While significant breakthroughs have been made in AI language ability, artificial visual intelligence, especially the ability to generate and simulate the world we see, is far lagging behind. To facilitate the development and accessibility of artificial visual intelligence, we created Open-Sora, an open-source video generation model designed to produce high-fidelity video content. Open-Sora supports a wide spectrum of visual generation tasks, including text-to-image generation, text-to-video generation, and image-to-video generation. The model leverages advanced deep learning architectures and training/inference techniques to enable flexible video synthesis, which could generate video content of up to 15 seconds, up to 720p resolution, and arbitrary aspect ratios. Specifically, we introduce Spatial-Temporal Diffusion Transformer (STDiT), an efficient diffusion framework for videos that decouples spatial and temporal attention. We also introduce a highly compressive 3D autoencoder to make representations compact and further accelerate training with an ad hoc training strategy. Through this initiative, we aim to foster innovation, creativity, and inclusivity within the community of AI content creation. By embracing the open-source principle, Open-Sora democratizes full access to all the training/inference/data preparation codes as well as model weights. All resources are publicly available at: https://github.com/hpcaitech/Open-Sora.

Key facts. Cited by 637 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 16 of its 38 references that are in the map. First posted on arXiv 29 Dec 2024.

Builds on

16 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  2. 2022Flow Matching for Generative Modeling6 Oct

Other references · 14

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  3. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  4. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  5. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  6. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  7. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  8. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  9. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 320
  10. 2022UL2: Unifying Language Learning Paradigmscited by 292
  11. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
  12. 2024FiT: Flexible Vision Transformer for Diffusion Modelcited by 84
2 more
  1. 2024Vript: A Video Is Worth Thousands of Wordscited by 71
  2. 2023A Theory on Adam Instability in Large-Scale Machine Learningcited by 54

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 345; 2026: 292.

637 papers.

Sort:

By area:

  1. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  2. 2025Improving Video Generation with Human Feedbackcited by 267
  3. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 218
  4. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  5. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  6. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 188
  7. 2025History-Guided Video Diffusioncited by 165
  8. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 163
  9. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
  10. 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 157
  11. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
  12. 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
38 more
  1. 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 126
  2. 2025GameFactory: Creating New Games with Generative Interactive Videoscited by 124
  3. 2025Video World Models with Long-term Spatial Memorycited by 124
  4. 2025Fast Video Generation with Sliding Tile Attentioncited by 123
  5. 2025SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inferencecited by 113
  6. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  7. 2025VSA: Faster Video Diffusion with Trainable Sparse Attentioncited by 100
  8. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformercited by 91
  9. 2025WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generationcited by 87
  10. 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 84
  11. 2025Automated Movie Generation via Multi-Agent CoT Planningcited by 82
  12. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 72
  13. 2026DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Controlcited by 71
  14. 2025WMPO: World Model-based Policy Optimization for Vision-Language-Action Modelscited by 68
  15. 2025Training-free and Adaptive Sparse Attention for Efficient Long Video Generationcited by 65
  16. 2025RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformerscited by 63
  17. 2025SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Trainingcited by 63
  18. 2025SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restorationcited by 62
  19. 2025InsViE-1M: Effective Instruction-based Video Editing with Elaborate Dataset Constructioncited by 62
  20. 2025VideoVLA: Video Generators Can Be Generalizable Robot Manipulatorscited by 61
  21. 2025MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidancecited by 60
  22. 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AIcited by 60
  23. 2025EnerVerse: Envisioning Embodied Future Space for Robotics Manipulationcited by 59
  24. 2026Helios: Real Real-Time Long Video Generation Modelcited by 56
  25. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
  26. 2026Motubrain: An Advanced World Action Model for Robot Controlcited by 54
  27. 2025AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusioncited by 53
  28. 2025PISA Experiments: Exploring Physics Post-Training for Video Diffusion Models by Watching Stuff Dropcited by 49
  29. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 49
  30. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 48
  31. 2026InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulationcited by 48
  32. 2026WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RLcited by 46
  33. 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Modelscited by 45
  34. 2025Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidancecited by 45
  35. 2026Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generationcited by 44
  36. 2025Pre-Trained Video Generative Models as World Simulatorscited by 43
  37. 2025InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generationcited by 43
  38. 2025ATI: Any Trajectory Instruction for Controllable Video Generationcited by 42

50 most cited of 637 All 637 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Video Is Worth a Thousand Images: Exploring the Latest Trends in Long Video Generation
  2. 2024DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation
  3. 2024EasyAnimate: High-Performance Video Generation Framework with Hybrid Windows Attention and Reward Backpropagation
  4. 2025Fine-Tuning Open Video Generators for Cinematic Scene Synthesis: A Small-Data Pipeline with LoRA and Wan2.1 I2V
  5. 2024WorldGPT: A Sora-Inspired Video AI Agent as Rich World Models from Text and Image Inputs
  6. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Models
  7. 2024Mora: Enabling Generalist Video Generation via A Multi-Agent Framework
  8. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applications