Evolution of Video Generative Foundations
On the map: World models → Video-generation world models and simulators
also in 1 more
Abstract (arXiv)
The rapid advancement of Artificial Intelligence Generated Content (AIGC) has revolutionized video generation, enabling systems ranging from proprietary pioneers like OpenAI's Sora, Google's Veo3, and Bytedance's Seedance to powerful open-source contenders like Wan and HunyuanVideo to synthesize temporally coherent and semantically rich videos. These advancements pave the way for building "world models" that simulate real-world dynamics, with applications spanning entertainment, education, and virtual reality. However, existing reviews on video generation often focus on narrow technical fields, e.g., Generative Adversarial Networks (GAN) and diffusion models, or specific tasks (e. g., video editing), lacking a comprehensive perspective on the field's evolution, especially regarding Auto-Regressive (AR) models and integration of multimodal information. To address these gaps, this survey firstly provides a systematic review of the development of video generation technology, tracing its evolution from early GANs to dominant diffusion models, and further to emerging AR-based and multimodal techniques. We conduct an in-depth analysis of the foundational principles, key advancements, and comparative strengths/limitations. Then, we explore emerging trends in multimodal video generation, emphasizing the integration of diverse data types to enhance contextual awareness. Finally, by bridging historical developments and contemporary innovations, this survey offers insights to guide future research in video generation and its applications, including virtual/augmented reality, personalized education, autonomous driving simulations, digital entertainment, and advanced world models, in this rapidly evolving field. For more details, please refer to the project at https://github.com/sjtuplayer/Awesome-Video-Foundations.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 272 of its 286 references that are in the map. First posted on arXiv 7 Apr 2026.
Builds on
272 of 286 references.
Closest ancestors · 19
Same direction, newest first
- 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
- 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
- 2025EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory1 Oct
- 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
- 2025Video World Models with Long-term Spatial Memory5 Jun
- 2025Learning 3D Persistent Embodied World Models5 May
- 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
- 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness27 Mar
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024World Model on Million-Length Video And Language With Blockwise RingAttention13 Feb
- 2023PEEKABOO: Interactive Video Generation via Masked-Diffusion12 Dec
7 more
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
- 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
- 2022Scalable Diffusion Models with Transformers19 Dec
- 2022MAGVIT: Masked Generative Video Transformer10 Dec
- 2022MaskGIT: Masked Generative Image Transformer8 Feb
- 2021VideoGPT: Video Generation using VQ-VAE and Transformers20 Apr
- 2020Denoising Diffusion Probabilistic Models19 Jun
Other references · 253
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023GPT-4 Technical Reportcited by 18,132
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2017Neural Discrete Representation Learningcited by 4,650
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
- 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
38 more
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2016Pixel Recurrent Neural Networkscited by 814
- 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
- 2023ModelScope Text-to-Video Technical Reportcited by 651
- 2024Autoregressive Image Generation without Vector Quantizationcited by 644
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
- 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
- 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
- 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
- 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
- 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
- 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 475
- 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
- 2025VACE: All-in-One Video Creation and Editingcited by 467
- 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
- 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
- 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
- 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2023Generative Multimodal Models are In-Context Learnerscited by 446
- 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
50 most cited of 253 All 253 (JSON)
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Datacited by 8
- 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generationcited by 5
- 2026Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attentioncited by 1
- 2026PhysWeep: Does a Video Generator Realize the Physics You Ask For?cited by 0
- 2026DiVid: Diagnosing Dimension-Specific Diversity Collapse in Video Generation Modelscited by 0
- 2026DiVid: Diagnosing Dimension-Specific Diversity Collapse in Video Generation Models1 Oct 2026
- 2026PhysWeep: Does a Video Generator Realize the Physics You Ask For?5 Sep 2026
- 2026Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attention16 Aug 2026
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation8 Jun 2026
- 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data1 Jun 2026
- 2026Toward Native Multimodal Modeling: A Roadmap25 May 2026
Similar, not cited
Close in wording, no citation either way
- 2024Video Is Worth a Thousand Images: Exploring the Latest Trends in Long Video Generation
- 2025Controllable Video Generation: A Survey
- 2026Generative AI for Video Trailer Synthesis: From Extractive Heuristics to Autoregressive Creativity
- 2025Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applications
- 2023Advancements in Generative AI: A Comprehensive Review of GANs, GPT, Autoencoders, Diffusion Model, and Transformers
- 2025Style Transfer: A Decade Survey
- 2024Open-Sora: Democratizing Efficient Video Production for All
- 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applications