# The Dawn of Video Generation: Preliminary Explorations with SORA-like Models

arXiv: https://arxiv.org/abs/2410.05227 · HTML: https://mlsanity.com/paper/2410.05227/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 114 references that are in the map.
- First posted on arXiv 7 Oct 2024.

## Builds on (most cited first, 30 of 62)

- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Show-o: One Single Transformer to Unify Multimodal Understanding and Generation](https://mlsanity.com/paper/2408.12528/) (2024)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [Autoregressive Image Generation without Vector Quantization](https://mlsanity.com/paper/2406.11838/) (2024)
- [Latte: Latent Diffusion Transformer for Video Generation](https://mlsanity.com/paper/2401.03048/) (2024)
- [Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model](https://mlsanity.com/paper/2408.11039/) (2024)
- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [CameraCtrl: Enabling Camera Control for Text-to-Video Generation](https://mlsanity.com/paper/2404.02101/) (2024)
- [LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models](https://mlsanity.com/paper/2309.15103/) (2023)
- [Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability](https://mlsanity.com/paper/2405.17398/) (2024)
- [OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation](https://mlsanity.com/paper/2407.02371/) (2024)
- [VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2303.08320/) (2023)
- [Photorealistic Video Generation with Diffusion Models](https://mlsanity.com/paper/2312.06662/) (2023)
- [Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2309.15818/) (2023)
- [Diffusion Models Are Real-Time Game Engines](https://mlsanity.com/paper/2408.14837/) (2024)
- [EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions](https://mlsanity.com/paper/2402.17485/) (2024)
- [A Survey on Video Diffusion Models](https://mlsanity.com/paper/2310.10647/) (2023)
- [DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation](https://mlsanity.com/paper/2403.06845/) (2024)
- [VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time](https://mlsanity.com/paper/2404.10667/) (2024)
- [T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation](https://mlsanity.com/paper/2407.14505/) (2024)
- [Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation](https://mlsanity.com/paper/2406.01900/) (2024)
- [Video Language Planning](https://mlsanity.com/paper/2310.10625/) (2023)
- [Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models](https://mlsanity.com/paper/2405.04233/) (2024)

## Built on by (most cited first, 30 of 31)

- [Improving Video Generation with Human Feedback](https://mlsanity.com/paper/2501.13918/) (2025)
- [VMBench: A Benchmark for Perception-Aligned Video Motion Generation](https://mlsanity.com/paper/2503.10076/) (2025)
- [Improved Video VAE for Latent Video Diffusion Model](https://mlsanity.com/paper/2411.06449/) (2024)
- [Video-T1: Test-Time Scaling for Video Generation](https://mlsanity.com/paper/2503.18942/) (2025)
- [Scaling Image and Video Generation via Test-Time Evolutionary Search](https://mlsanity.com/paper/2505.17618/) (2025)
- [DisPose: Disentangling Pose Guidance for Controllable Human Image Animation](https://mlsanity.com/paper/2412.09349/) (2024)
- [DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models](https://mlsanity.com/paper/2506.03517/) (2025)
- [VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Step](https://mlsanity.com/paper/2504.01956/) (2025)
- [VideoGen-Eval: Agent-based System for Video Generation Evaluation](https://mlsanity.com/paper/2503.23452/) (2025)
- [FingER: Content Aware Fine-grained Evaluation with Reasoning for AI-Generated Videos](https://mlsanity.com/paper/2504.10358/) (2025)
- [GAS: Generative Avatar Synthesis from a Single Image](https://mlsanity.com/paper/2502.06957/) (2025)
- [Impossible Videos](https://mlsanity.com/paper/2503.14378/) (2025)
- [Video Generation Models Are Good Latent Reward Models](https://mlsanity.com/paper/2511.21541/) (2025)
- [HERO: Human Reaction Generation from Videos](https://mlsanity.com/paper/2503.08270/) (2025)
- [SIGMAN:Scaling 3D Human Gaussian Generation with Millions of Assets](https://mlsanity.com/paper/2504.06982/) (2025)
- [VR-Thinker: Boosting Video Reward Models through Thinking-with-Image Reasoning](https://mlsanity.com/paper/2510.10518/) (2025)
- [LPM 1.0: Video-based Character Performance Model](https://mlsanity.com/paper/2604.07823/) (2026)
- [ETVA: Evaluation of Text-to-Video Alignment via Fine-grained Question Generation and Answering](https://mlsanity.com/paper/2503.16867/) (2025)
- [AniSora: Exploring the Frontiers of Animation Video Generation in the Sora Era](https://mlsanity.com/paper/2412.10255/) (2024)
- [UVE: Are MLLMs Unified Evaluators for AI-Generated Videos?](https://mlsanity.com/paper/2503.09949/) (2025)
- [BrokenVideos: A Benchmark Dataset for Fine-Grained Artifact Localization in AI-Generated Videos](https://mlsanity.com/paper/2506.20103/) (2025)
- [PatchVSR: Breaking Video Diffusion Resolution Limits with Patch-wise Video Super-Resolution](https://mlsanity.com/paper/2509.26025/) (2025)
- [Mind the Generative Details: Direct Localized Detail Preference Optimization for Video Diffusion Models](https://mlsanity.com/paper/2601.04068/) (2026)
- [Gloria: Consistent Character Video Generation via Content Anchors](https://mlsanity.com/paper/2603.29931/) (2026)
- [AesRM: Improving Video Aesthetics with Expert-Level Feedback](https://mlsanity.com/paper/2604.28078/) (2026)
- [Think, then Score: Decoupled Reasoning and Scoring for Video Reward Modeling](https://mlsanity.com/paper/2605.05922/) (2026)
- [UniMMVSR: A Unified Multi-Modal Framework for Cascaded Video Super-Resolution](https://mlsanity.com/paper/2510.08143/) (2025)
- [End-to-End Spatial-Temporal Transformer for Real-time 4D HOI Reconstruction](https://mlsanity.com/paper/2603.14435/) (2026)
- [LogiShot: Logically Coherent Cross-Shot Video Generation](https://mlsanity.com/paper/2608.08820/) (2026)
- [Tera-MIND: Tera-scale mouse brain simulation via spatial mRNA-guided diffusion](https://mlsanity.com/paper/2503.01220/) (2025)

## Abstract (arXiv)

High-quality video generation, encompassing text-to-video (T2V), image-to-video (I2V), and video-to-video (V2V) generation, holds considerable significance in content creation to benefit anyone express their inherent creativity in new ways and world simulation to modeling and understanding the world. Models like SORA have advanced generating videos with higher resolution, more natural motion, better vision-language alignment, and increased controllability, particularly for long video sequences. These improvements have been driven by the evolution of model architectures, shifting from UNet to more scalable and parameter-rich DiT models, along with large-scale data expansion and refined training strategies. However, despite the emergence of DiT-based closed-source and open-source models, a comprehensive investigation into their capabilities and limitations remains lacking. Furthermore, the rapid development has made it challenging for recent benchmarks to fully cover SORA-like models and recognize their significant advancements. Additionally, evaluation metrics often fail to align with human preferences.