# A Survey on Long Video Generation: Challenges, Methods, and Prospects

arXiv: https://arxiv.org/abs/2403.16407 · HTML: https://mlsanity.com/paper/2403.16407/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 48 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 30 of 38)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023)
- [Phenaki: Variable Length Video Generation From Open Domain Textual Description](https://mlsanity.com/paper/2210.02399/) (2022)
- [A Short Note about Kinetics-600](https://mlsanity.com/paper/1808.01340/) (2018)
- [VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2303.08320/) (2023)
- [MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation](https://mlsanity.com/paper/2205.09853/) (2022)
- [StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2](https://mlsanity.com/paper/2112.14683/) (2021)
- [Flexible Diffusion Modeling of Long Videos](https://mlsanity.com/paper/2205.11495/) (2022)
- [Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer](https://mlsanity.com/paper/2204.03638/) (2022)
- [SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction](https://mlsanity.com/paper/2310.20700/) (2023)
- [Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks](https://mlsanity.com/paper/2202.10571/) (2022)
- [Video Probabilistic Diffusion Models in Projected Latent Space](https://mlsanity.com/paper/2302.07685/) (2023)
- [NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation](https://mlsanity.com/paper/2303.12346/) (2023)
- [VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning](https://mlsanity.com/paper/2309.15091/) (2023)
- [Make Pixels Dance: High-Dynamic Video Generation](https://mlsanity.com/paper/2311.10982/) (2023)
- [Self-Supervised Visual Planning with Temporal Skip Connections](https://mlsanity.com/paper/1710.05268/) (2017)
- [Diffusion Models for Video Prediction and Infilling](https://mlsanity.com/paper/2206.07696/) (2022)
- [Consistent View Synthesis with Pose-Guided Diffusion Models](https://mlsanity.com/paper/2303.17598/) (2023)
- [Generating Long Videos of Dynamic Scenes](https://mlsanity.com/paper/2206.03429/) (2022)
- [Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks](https://mlsanity.com/paper/1709.07592/) (2017)

## Built on by (most cited first, 30 of 40)

- [FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention](https://mlsanity.com/paper/2407.19918/) (2024)
- [PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation](https://mlsanity.com/paper/2412.00596/) (2024)
- [ATLAS: Learning to Optimally Memorize the Context at Test Time](https://mlsanity.com/paper/2505.23735/) (2025)
- [It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization](https://mlsanity.com/paper/2504.13173/) (2025)
- [Video-T1: Test-Time Scaling for Video Generation](https://mlsanity.com/paper/2503.18942/) (2025)
- [Exploring the Evolution of Physics Cognition in Video Generation: A Survey](https://mlsanity.com/paper/2503.21765/) (2025)
- [Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation](https://mlsanity.com/paper/2412.16211/) (2024)
- [CineTrans: Learning to Generate Videos with Cinematic Transitions via Masked Diffusion Models](https://mlsanity.com/paper/2508.11484/) (2025)
- [A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applications](https://mlsanity.com/paper/2410.15595/) (2024)
- [Code2Video: A Code-centric Paradigm for Educational Video Generation](https://mlsanity.com/paper/2510.01174/) (2025)
- [A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insights](https://mlsanity.com/paper/2407.08428/) (2024)
- [WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World](https://mlsanity.com/paper/2512.10958/) (2025)
- [SurGen: Text-Guided Diffusion Model for Surgical Video Generation](https://mlsanity.com/paper/2408.14028/) (2024)
- [MiLA: Multi-view Intensive-fidelity Long-term Video Generation World Model for Autonomous Driving](https://mlsanity.com/paper/2503.15875/) (2025)
- [Generative AI for Cel-Animation: A Survey](https://mlsanity.com/paper/2501.06250/) (2025)
- [LongDiff: Training-Free Long Video Generation in One Go](https://mlsanity.com/paper/2503.18150/) (2025)
- [Mode Seeking meets Mean Seeking for Fast Long Video Generation](https://mlsanity.com/paper/2602.24289/) (2026)
- [Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook](https://mlsanity.com/paper/2411.19537/) (2024)
- [FastCar: Cache Attentive Replay for Fast Auto-Regressive Video Generation on the Edge](https://mlsanity.com/paper/2505.14709/) (2025)
- [Video-GPT via Next Clip Diffusion](https://mlsanity.com/paper/2505.12489/) (2025)
- [Video Is Worth a Thousand Images: Exploring the Latest Trends in Long Video Generation](https://mlsanity.com/paper/2412.18688/) (2024)
- [Evolution of Video Generative Foundations](https://mlsanity.com/paper/2604.06339/) (2026)
- [ShotDirector: Directorially Controllable Multi-Shot Video Generation with Cinematographic Transitions](https://mlsanity.com/paper/2512.10286/) (2025)
- [WaveStitch: Flexible and Fast Conditional Time Series Generation with Diffusion Models](https://mlsanity.com/paper/2503.06231/) (2025)
- [A Survey: Spatiotemporal Consistency in Video Generation](https://mlsanity.com/paper/2502.17863/) (2025)
- [GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusion](https://mlsanity.com/paper/2605.12957/) (2026)
- [Cognitively Inspired Energy-Based World Models](https://mlsanity.com/paper/2406.08862/) (2024)
- [A Self-supervised Motion Representation for Portrait Video Generation](https://mlsanity.com/paper/2503.10096/) (2025)
- [Frame-Level Captions for Long Video Generation with Complex Multi Scenes](https://mlsanity.com/paper/2505.20827/) (2025)
- [SSG-Dit: A Spatial Signal Guided Framework for Controllable Video Generation](https://mlsanity.com/paper/2508.17062/) (2025)

## Abstract (arXiv)

Video generation is a rapidly advancing research area, garnering significant attention due to its broad range of applications. One critical aspect of this field is the generation of long-duration videos, which presents unique challenges and opportunities. This paper presents the first survey of recent advancements in long video generation and summarises them into two key paradigms: divide and conquer temporal autoregressive. We delve into the common models employed in each paradigm, including aspects of network design and conditioning techniques. Furthermore, we offer a comprehensive overview and classification of the datasets and evaluation metrics which are crucial for advancing long video generation research. Concluding with a summary of existing studies, we also discuss the emerging challenges and future directions in this dynamic field. We hope that this survey will serve as an essential reference for researchers and practitioners in the realm of long video generation.