mlsanityData to 14 Oct 2026

A Survey on Video Diffusion Models

arXiv 2310.10647 ↗First posted 16 Oct 2023cs.CV cs.AI cs.LGcited by 206

On the map: Diffusion models → Video generation

Abstract (arXiv)

The recent wave of AI-generated content (AIGC) has witnessed substantial success in computer vision, with the diffusion model playing a crucial role in this achievement. Due to their impressive generative capabilities, diffusion models are gradually superseding methods based on GANs and auto-regressive Transformers, demonstrating exceptional performance not only in image generation and editing, but also in the realm of video-related research. However, existing surveys mainly focus on diffusion models in the context of image generation, with few up-to-date reviews on their application in the video domain. To address this gap, this paper presents a comprehensive review of video diffusion models in the AIGC era. Specifically, we begin with a concise introduction to the fundamentals and evolution of diffusion models. Subsequently, we present an overview of research on diffusion models in the video domain, categorizing the work into three key areas: video generation, video editing, and other video understanding tasks. We conduct a thorough review of the literature in these three key areas, including further categorization and practical contributions in the field. Finally, we discuss the challenges faced by research in this domain and outline potential future developmental trends. A comprehensive list of video diffusion models studied in this survey is available at https://github.com/ChenHsing/Awesome-Video-Diffusion-Models.

Key facts. Cited by 206 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 243 of its 326 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

243 of 326 references.

Closest ancestors · 56

Same direction, newest first

  1. 2023FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editing9 Oct
  2. 2023Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Models2 Oct
  3. 2023LLM-grounded Video Diffusion Models29 Sep
  4. 2023CCEdit: Creative and Controllable Video Editing via Diffusion Models28 Sep
  5. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation27 Sep
  6. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models26 Sep
  7. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generation7 Sep
  8. 2023MagicProp: Diffusion-based Video Editing via Motion-aware Appearance Propagation2 Sep
  9. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation1 Sep
  10. 2023EVE: Efficient zero-shot text-based Video Editing with Depth Map Guidance and Temporal Consistency Constraints21 Aug
  11. 2023MeDM: Mediating Image Diffusion Models for Video-to-Video Translation with Temporal Correspondence Guidance19 Aug
  12. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editing18 Aug
38 more
  1. 2023Edit Temporal-Consistent Videos with Image Diffusion Model17 Aug
  2. 2023Dual-Stream Diffusion Net for Text-to-Video Generation16 Aug
  3. 2023ModelScope Text-to-Video Technical Report12 Aug
  4. 2023DiffSynth: Latent In-Iteration Deflickering for Realistic Video Synthesis7 Aug
  5. 2023VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNet26 Jul
  6. 2023InFusion: Inject and Attention Fusion for Multi Concept Zero-Shot Text-based Video Editing22 Jul
  7. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  8. 2023GD-VDM: Generated Depth for better Diffusion-based Video Generation19 Jun
  9. 2023VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editing14 Jun
  10. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation13 Jun
  11. 2023Video Diffusion Models with Local-Global Context Guidance5 Jun
  12. 2023SAVE: Spectral-Shift-Aware Adaptation of Image Diffusion Models for Text-driven Video Editing30 May
  13. 2023Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising29 May
  14. 2023Towards Consistent Video Editing with Text-to-Image Diffusion Models27 May
  15. 2023ControlVideo: Training-free Controllable Text-to-Video Generation22 May
  16. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models17 May
  17. 2023Make-A-Protagonist: Generic Video Editing with An Ensemble of Experts15 May
  18. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesis27 Apr
  19. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  20. 2023Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generation17 Apr
  21. 2023Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Models30 Mar
  22. 2023DiffTAD: Temporal Action Detection with Proposal Denoising Diffusion27 Mar
  23. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Models24 Mar
  24. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators23 Mar
  25. 2023Pix2Video: Video Editing using Image Diffusion22 Mar
  26. 2023NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation22 Mar
  27. 2023DiffusionRet: Generative Text-Video Retrieval with Diffusion Model17 Mar
  28. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editing16 Mar
  29. 2023LDMVFI: Video Frame Interpolation with Latent Diffusion Models16 Mar
  30. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation15 Mar
  31. 2023Spatial-temporal Transformer-guided Diffusion based Data Augmentation for Efficient Skeleton-based Action Recognition26 Feb
  32. 2023Video Probabilistic Diffusion Models in Projected Latent Space15 Feb
  33. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
  34. 2023Dreamix: Video Diffusion Models are General Video Editors2 Feb
  35. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
  36. 2022VIDM: Video Implicit Diffusion Models1 Dec
  37. 2022SinFusion: Training Diffusion Models on a Single Image or Video21 Nov
  38. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models20 Nov

50 newest of 56 All 56 (JSON)

Other references · 187

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2023GPT-4 Technical Reportcited by 18,396
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  8. 2022Training language models to follow instructions with human feedbackcited by 15,971
  9. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  10. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  11. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
38 more
  1. 2023Segment Anythingcited by 8,234
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2022Scalable Diffusion Models with Transformerscited by 6,054
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2022Generative Adversarial Networkscited by 5,627
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  13. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  14. 2017Neural Discrete Representation Learningcited by 4,712
  15. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  16. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  17. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  18. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  19. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  20. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  21. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  22. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  23. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  24. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  25. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  26. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  27. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  28. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  29. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  30. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  31. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  32. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  33. 2021ViViT: A Video Vision Transformercited by 1,620
  34. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  35. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  36. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  37. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  38. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420

50 most cited of 187 All 187 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 66; 2025: 74; 2026: 55.

206 papers.

Sort:

By area:

  1. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  2. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  3. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  4. 2023Video Understanding with Large Language Models: A Surveycited by 206
  5. 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
  6. 2023Rich Human Feedback for Text-to-Image Generationcited by 141
  7. 2024StableAnimator: High-Quality Identity-Preserving Human Image Animationcited by 88
  8. 2024Video as the New Language for Real-World Decision Makingcited by 87
  9. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 83
  10. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 73
  11. 2025MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidancecited by 61
  12. 2024CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generationcited by 60
38 more
  1. 2023MotionEditor: Editing Video Motion via Content-Aware Diffusioncited by 57
  2. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributionscited by 53
  3. 2023Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understandingcited by 46
  4. 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 45
  5. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  6. 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generationcited by 44
  7. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 42
  8. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  9. 2024DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenescited by 35
  10. 2025Efficient Diffusion Models: A Surveycited by 35
  11. 2025MagicMirror: ID-Preserved Video Generation in Video Diffusion Transformerscited by 34
  12. 2024Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generationcited by 32
  13. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
  14. 2024TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Modelscited by 31
  15. 2024The Dawn of Video Generation: Preliminary Explorations with SORA-like Modelscited by 31
  16. 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
  17. 2024MuseTalk: Real-Time High-Fidelity Video Dubbing via Spatio-Temporal Samplingcited by 28
  18. 2024Wasserstein Flow Matching: Generative modeling over families of distributionscited by 28
  19. 2024From Sora What We Can See: A Survey of Text-to-Video Generationcited by 27
  20. 2024Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video Diffusion Modelscited by 27
  21. 2024A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insightscited by 25
  22. 2024Matten: Video Generation with Mamba-Attentioncited by 24
  23. 2025Generative Physical AI in Vision: A Surveycited by 24
  24. 20253D Scene Generation: A Surveycited by 24
  25. 2024Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agendacited by 22
  26. 2025NarrLV: Towards a Comprehensive Narrative-Centric Evaluation for Long Video Generationcited by 22
  27. 2024AID: Adapting Image2Video Diffusion Models for Instruction-guided Video Predictioncited by 21
  28. 2024HOI-Swap: Swapping Objects in Videos with Hand-Object Interaction Awarenesscited by 21
  29. 2024Warped Diffusion: Solving Video Inverse Problems with Image Diffusion Modelscited by 21
  30. 2023AdaDiff: Adaptive Step Selection for Fast Diffusion Modelscited by 20
  31. 2025Error Analyses of Auto-Regressive Video Diffusion Models: A Unified Frameworkcited by 20
  32. 2025A Survey on Video Temporal Grounding with Multimodal Large Language Modelcited by 20
  33. 2023VIDiff: Translating Videos via Multi-Modal Instructions with Diffusion Modelscited by 19
  34. 2024ImageFlowNet: Forecasting Multiscale Image-Level Trajectories of Disease Progression with Irregularly-Sampled Longitudinal Medical Imagescited by 19
  35. 2025TR-DQ: Time-Rotation Diffusion Quantizationcited by 19
  36. 2025Sculpting Memory: Multi-Concept Forgetting in Diffusion Models via Dynamic Mask and Concept-Aware Optimizationcited by 19
  37. 2025FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusioncited by 19
  38. 2025AniMaker: Multi-Agent Animated Storytelling with MCTS-Driven Clip Generationcited by 19

50 most cited of 206 All 206 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applications
  2. 2023State of the Art on Diffusion Models for Visual Computing
  3. 2024Video Diffusion Models: A Survey
  4. 2024Accelerating Video Diffusion Models via Distribution Matching
  5. 2025From Image to Video: An Empirical Study of Diffusion Representations
  6. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation
  7. 2026Efficient Video Diffusion Models: Advancements and Challenges
  8. 2024Diffusion Models in Low-Level Vision: A Survey