mlsanityData to 13 Oct 2026

The Dawn of Video Generation: Preliminary Explorations with SORA-like Models

arXiv 2410.05227 ↗First posted 7 Oct 2024cs.CVcited by 30

Not on the map. Why?

Abstract (arXiv)

High-quality video generation, encompassing text-to-video (T2V), image-to-video (I2V), and video-to-video (V2V) generation, holds considerable significance in content creation to benefit anyone express their inherent creativity in new ways and world simulation to modeling and understanding the world. Models like SORA have advanced generating videos with higher resolution, more natural motion, better vision-language alignment, and increased controllability, particularly for long video sequences. These improvements have been driven by the evolution of model architectures, shifting from UNet to more scalable and parameter-rich DiT models, along with large-scale data expansion and refined training strategies. However, despite the emergence of DiT-based closed-source and open-source models, a comprehensive investigation into their capabilities and limitations remains lacking. Furthermore, the rapid development has made it challenging for recent benchmarks to fully cover SORA-like models and recognize their significant advancements. Additionally, evaluation metrics often fail to align with human preferences.

Key facts. Cited by 30 papers in the map (data through 13 Oct 2026). Builds on 62 of its 114 references that are in the map. First posted on arXiv 7 Oct 2024.

Builds on

62 of 114 references.

References · 62

  1. 2022Flow Matching for Generative Modelingcited by 5,542
  2. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,607
  3. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,550
  4. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,218
  5. 2023Consistency Modelscited by 1,797
  6. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 975
  7. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 931
  8. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 740
  9. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 680
  10. 2023ModelScope Text-to-Video Technical Reportcited by 655
  11. 2024Autoregressive Image Generation without Vector Quantizationcited by 647
  12. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 539
38 more
  1. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 475
  2. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 473
  3. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 451
  4. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  5. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 341
  6. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 334
  7. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 331
  8. 2023Photorealistic Video Generation with Diffusion Modelscited by 305
  9. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 301
  10. 2024Diffusion Models Are Real-Time Game Enginescited by 286
  11. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
  12. 2023A Survey on Video Diffusion Modelscited by 206
  13. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 189
  14. 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 186
  15. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 178
  16. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 173
  17. 2023Video Language Planningcited by 161
  18. 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Modelscited by 160
  19. 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generationcited by 123
  20. 2024Video Diffusion Alignment via Reward Gradientscited by 102
  21. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  22. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 91
  23. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokenscited by 88
  24. 2024Pandora: Towards General World Model with Natural Language Actions and Video Statescited by 84
  25. 2023LLM-grounded Video Diffusion Modelscited by 83
  26. 2024EasyAnimate: High-Performance Video Generation Framework with Hybrid Windows Attention and Reward Backpropagationcited by 76
  27. 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
  28. 2024ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generationcited by 74
  29. 2024ToonCrafter: Generative Cartoon Interpolationcited by 68
  30. 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
  31. 2024HumanVid: Demystifying Training Data for Camera-controllable Human Image Animationcited by 63
  32. 2024Magic-Me: Identity-Specific Video Customized Diffusioncited by 59
  33. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generationcited by 53
  34. 2024MagicTime: Time-lapse Video Generation Models as Metamorphic Simulatorscited by 51
  35. 2024VideoTetris: Towards Compositional Text-to-Video Generationcited by 51
  36. 2024VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesiscited by 49
  37. 2024TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generationcited by 42
  38. 2024CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attentioncited by 39

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 3; 2025: 21; 2026: 6.

30 papers.

Sort:

By area:

  1. 2025Improving Video Generation with Human Feedbackcited by 272
  2. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 48
  3. 2024Improved Video VAE for Latent Video Diffusion Modelcited by 46
  4. 2025Video-T1: Test-Time Scaling for Video Generationcited by 39
  5. 2025Scaling Image and Video Generation via Test-Time Evolutionary Searchcited by 39
  6. 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animationcited by 35
  7. 2025DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Modelscited by 32
  8. 2025VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Stepcited by 27
  9. 2025VideoGen-Eval: Agent-based System for Video Generation Evaluationcited by 23
  10. 2025FingER: Content Aware Fine-grained Evaluation with Reasoning for AI-Generated Videoscited by 22
  11. 2025GAS: Generative Avatar Synthesis from a Single Imagecited by 19
  12. 2025Impossible Videoscited by 16
18 more
  1. 2025HERO: Human Reaction Generation from Videoscited by 15
  2. 2025Video Generation Models Are Good Latent Reward Modelscited by 15
  3. 2025SIGMAN:Scaling 3D Human Gaussian Generation with Millions of Assetscited by 12
  4. 2025VR-Thinker: Boosting Video Reward Models through Thinking-with-Image Reasoningcited by 12
  5. 2026LPM 1.0: Video-based Character Performance Modelcited by 11
  6. 2024AniSora: Exploring the Frontiers of Animation Video Generation in the Sora Eracited by 8
  7. 2025ETVA: Evaluation of Text-to-Video Alignment via Fine-grained Question Generation and Answeringcited by 8
  8. 2025UVE: Are MLLMs Unified Evaluators for AI-Generated Videos?cited by 6
  9. 2025BrokenVideos: A Benchmark Dataset for Fine-Grained Artifact Localization in AI-Generated Videoscited by 4
  10. 2025PatchVSR: Breaking Video Diffusion Resolution Limits with Patch-wise Video Super-Resolutioncited by 4
  11. 2026Mind the Generative Details: Direct Localized Detail Preference Optimization for Video Diffusion Modelscited by 3
  12. 2026Gloria: Consistent Character Video Generation via Content Anchorscited by 3
  13. 2026Think, then Score: Decoupled Reasoning and Scoring for Video Reward Modelingcited by 2
  14. 2025UniMMVSR: A Unified Multi-Modal Framework for Cascaded Video Super-Resolutioncited by 1
  15. 2026End-to-End Spatial-Temporal Transformer for Real-time 4D HOI Reconstructioncited by 1
  16. 2026LogiShot: Logically Coherent Cross-Shot Video Generationcited by 1
  17. 2025Tera-MIND: Tera-scale mouse brain simulation via spatial mRNA-guided diffusioncited by 0
  18. 2025MUG-V 10B: High-efficiency Training Pipeline for Large Video Generation Modelscited by 0