mlsanityData to 7 Oct 2026

Latent Video Diffusion Models for High-Fidelity Long Video Generation

arXiv 2211.13221 ↗First posted 23 Nov 2022cs.CV cs.AIcited by 385

On the map: Diffusion models → Video generation

Abstract (arXiv)

AI-generated content has attracted lots of attention recently, but photo-realistic video synthesis is still challenging. Although many attempts using GANs and autoregressive models have been made in this area, the visual quality and length of generated videos are far from satisfactory. Diffusion models have shown remarkable results recently but require significant computational resources. To address this, we introduce lightweight video diffusion models by leveraging a low-dimensional 3D latent space, significantly outperforming previous pixel-space video diffusion models under a limited computational budget. In addition, we propose hierarchical diffusion in the latent space such that longer videos with more than one thousand frames can be produced. To further overcome the performance degradation issue for long video generation, we propose conditional latent perturbation and unconditional guidance that effectively mitigate the accumulated errors during the extension of video length. Extensive experiments on small domain datasets of different categories suggest that our framework generates more realistic and longer videos than previous strong baselines. We additionally provide an extension to large-scale text-to-video generation to demonstrate the superiority of our work. Our code and models will be made publicly available.

Key facts. Cited by 385 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 45 of its 47 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

45 of 47 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models20 Nov
  2. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  3. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation19 May
  4. 2022Video Diffusion Models7 Apr

Other references · 41

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
29 more
  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  7. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  8. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  9. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  10. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  11. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  12. 2020First Order Motion Model for Image Animationcited by 698
  13. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  14. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  15. 2016Generating Videos with Scene Dynamicscited by 376
  16. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  17. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  18. 2016Temporal Generative Adversarial Nets with Singular Value Clippingcited by 198
  19. 2019Scaling Autoregressive Video Modelscited by 189
  20. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  21. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162
  22. 2018Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GANcited by 120
  23. 2022Generating Long Videos of Dynamic Scenescited by 113
  24. 2020Latent Video Transformercited by 106
  25. 2017Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networkscited by 88
  26. 2019VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generationcited by 74
  27. 2021Predicting Video with VQVAEcited by 61
  28. 2018Probabilistic Video Generation using Holistic Attribute Controlcited by 24
  29. 2021Hierarchical Video Generation for Complex Datacited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 26; 2024: 119; 2025: 177; 2026: 63.

385 papers.

Sort:

By area:

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  2. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  3. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  4. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  5. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  6. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  7. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  8. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  9. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  10. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  11. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 231
  12. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
38 more
  1. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 190
  2. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
  3. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
  4. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 177
  5. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
  6. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Modelscited by 172
  7. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 172
  8. 2025History-Guided Video Diffusioncited by 165
  9. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
  10. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 153
  11. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  12. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  13. 2024FIFO-Diffusion: Generating Infinite Videos from Text without Trainingcited by 134
  14. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 126
  15. 2025Video World Models with Long-term Spatial Memorycited by 124
  16. 2023FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generationcited by 119
  17. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
  18. 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
  19. 2025WorldModelBench: Judging Video Generation Models As World Modelscited by 116
  20. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 115
  21. 2025Long Context Tuning for Video Generationcited by 110
  22. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  23. 2023DemoFusion: Democratising High-Resolution Image Generation With No cited by 105
  24. 2024Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesiscited by 105
  25. 2023VDT: General-purpose Video Diffusion Transformers via Mask Modelingcited by 102
  26. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 102
  27. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 100
  28. 2025One-Minute Video Generation with Test-Time Trainingcited by 100
  29. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  30. 2024IRASim: A Fine-Grained World Model for Robot Manipulationcited by 93
  31. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 93
  32. 2025Controllable Video Generation: A Surveycited by 93
  33. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedbackcited by 85
  34. 2025Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuningcited by 81
  35. 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 75
  36. 2024DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generationcited by 73
  37. 2023I2V-Adapter: A General Image-to-Video Adapter for Diffusion Modelscited by 70
  38. 2024Vlogger: Make Your Dream A Vlogcited by 70

50 most cited of 385 All 385 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition
  2. 2023Video Probabilistic Diffusion Models in Projected Latent Space
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
  4. 2023A Survey on Video Diffusion Models
  5. 2024Accelerating Video Diffusion Models via Distribution Matching
  6. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation
  7. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models
  8. 2025VGDFR: Diffusion-based Video Generation with Dynamic Latent Frame Rate