mlsanityData to 7 Oct 2026

Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models

arXiv 2402.17177 ↗First posted 27 Feb 2024cs.CV cs.AI cs.LGcited by 467

Not on the map. Why?

Cited by 166 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Sora is a text-to-video generative AI model, released by OpenAI in February 2024. The model is trained to generate videos of realistic or imaginative scenes from text instructions and show potential in simulating the physical world. Based on public technical reports and reverse engineering, this paper presents a comprehensive review of the model's background, related technologies, applications, remaining challenges, and future directions of text-to-video AI models. We first trace Sora's development and investigate the underlying technologies used to build this "world simulator". Then, we describe in detail the applications and potential impact of Sora in multiple industries ranging from film-making and education to marketing. We discuss the main challenges and limitations that need to be addressed to widely deploy Sora, such as ensuring safe and unbiased video generation. Lastly, we discuss the future development of Sora and video generation models in general, and how advancements in the field could enable new ways of human-AI interaction, boosting productivity and creativity of video generation.

Key facts. Cited by 467 papers in the map (data through 7 Oct 2026). Builds on 166 of its 192 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

166 of 192 references.

References · 166

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  7. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  9. 2023GPT-4 Technical Reportcited by 18,132
  10. 2022Training language models to follow instructions with human feedbackcited by 15,712
  11. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
38 more
  1. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2022Scalable Diffusion Models with Transformerscited by 5,931
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  12. 2017Deep reinforcement learning from human preferencescited by 3,846
  13. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  14. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  15. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  16. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  17. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  18. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  19. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  20. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  21. 2022Visual Prompt Tuningcited by 1,674
  22. 2021ViViT: A Video Vision Transformercited by 1,610
  23. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  24. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  25. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  26. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  27. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  28. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  29. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  30. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  31. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  32. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  33. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  34. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  35. 2022Token Merging: Your ViT But Fastercited by 995
  36. 2021Scaling Vision Transformerscited by 993
  37. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  38. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958

50 most cited of 166 All 166 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 132; 2025: 201; 2026: 134.

467 papers.

Sort:

By area:

  1. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  2. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  3. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 215
  4. 2024Large Language Models for Education: A Survey and Outlookcited by 185
  5. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
  6. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 168
  7. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 141
  8. 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 121
  9. 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 112
  10. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  11. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 100
  12. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
38 more
  1. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 85
  2. 2025HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generationcited by 84
  3. 2024GFT: Graph Foundation Model with Transferable Tree Vocabularycited by 82
  4. 2025Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuningcited by 81
  5. 2025Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Trainingcited by 78
  6. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformerscited by 77
  7. 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
  8. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  9. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  10. 2025Stable Video Infinity: Infinite-Length Video Generation with Error Recyclingcited by 68
  11. 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
  12. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulatorscited by 65
  13. 2025SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restorationcited by 62
  14. 2024Gradient Guidance for Diffusion Models: An Optimization Perspectivecited by 59
  15. 2025Vidar: Embodied Video Diffusion Model for Generalist Manipulationcited by 58
  16. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Taskscited by 57
  17. 2025DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generationcited by 56
  18. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generationcited by 53
  19. 2024Endora: Video Generation Models as Endoscopy Simulatorscited by 53
  20. 2024LT3SD: Latent Trees for 3D Scene Diffusioncited by 53
  21. 2024Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matchingcited by 52
  22. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  23. 2024Mora: Enabling Generalist Video Generation via A Multi-Agent Frameworkcited by 41
  24. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 39
  25. 2025Video-T1: Test-Time Scaling for Video Generationcited by 39
  26. 2025FlexWorld: Progressively Expanding 3D Scenes for Flexiable-View Synthesiscited by 37
  27. 2025VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understandingcited by 32
  28. 2024GAIA: Rethinking Action Quality Assessment for AI-Generated Videoscited by 30
  29. 2024The Dawn of Video Generation: Preliminary Explorations with SORA-like Modelscited by 30
  30. 2025Emergent Temporal Correspondences from Video Diffusion Transformerscited by 30
  31. 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 29
  32. 2024FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Expertscited by 27
  33. 2024MarS: a Financial Market Simulation Engine Powered by Generative Foundation Modelcited by 27
  34. 2024Pathways on the Image Manifold: Image Editing via Video Generationcited by 27
  35. 2024From Sora What We Can See: A Survey of Text-to-Video Generationcited by 26
  36. 2025VISTA: A Test-Time Self-Improving Video Generation Agentcited by 26
  37. 2025DexGrasp Anything: Towards Universal Robotic Dexterous Grasping with Physics Awarenesscited by 25
  38. 2025NormalCrafter: Learning Temporally Consistent Normals from Video Diffusion Priorscited by 25

50 most cited of 467 All 467 (JSON)