mlsanityData to 7 Oct 2026

VIDM: Video Implicit Diffusion Models

arXiv 2212.00235 ↗First posted 1 Dec 2022cs.CVcited by 78

On the map: Diffusion models → Video generation

Abstract (arXiv)

Diffusion models have emerged as a powerful generative method for synthesizing high-quality and diverse set of images. In this paper, we propose a video generation method based on diffusion models, where the effects of motion are modeled in an implicit condition manner, i.e. one can sample plausible video motions according to the latent feature of frames. We improve the quality of the generated videos by proposing multiple strategies such as sampling space truncation, robustness penalty, and positional group normalization. Various experiments are conducted on datasets consisting of videos with different resolutions and different number of frames. Results show that the proposed method outperforms the state-of-the-art generative adversarial network-based methods by a significant margin in terms of FVD scores as well as perceptible visual quality.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 60 of its 68 references that are in the map. First posted on arXiv 1 Dec 2022.

Builds on

60 of 68 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Flexible Diffusion Modeling of Long Videos23 May
  2. 2022Video Diffusion Models7 Apr

Other references · 58

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2013Auto-Encoding Variational Bayescited by 3,980
  11. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  12. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
38 more
  1. 2016Improved Techniques for Training GANscited by 2,938
  2. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  3. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  4. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  5. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  6. 2014Conditional Generative Adversarial Netscited by 2,152
  7. 2016Density estimation using Real NVPcited by 2,029
  8. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  9. 2017The Kinetics Human Action Video Datasetcited by 1,894
  10. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  11. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  12. 2018Group Normalizationcited by 1,514
  13. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  14. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  15. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  16. 2016Pixel Recurrent Neural Networkscited by 814
  17. 2020First Order Motion Model for Image Animationcited by 698
  18. 2018Which Training Methods for GANs do actually Converge?cited by 672
  19. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  20. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  21. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
  22. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  23. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
  24. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  25. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 421
  26. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  27. 2016Generating Videos with Scene Dynamicscited by 376
  28. 2016A Learned Representation For Artistic Stylecited by 340
  29. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
  30. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  31. 2018cGANs with Projection Discriminatorcited by 253
  32. 2021Deblurring via Stochastic Refinementcited by 248
  33. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  34. 2016Temporal Generative Adversarial Nets with Singular Value Clippingcited by 198
  35. 2019Scaling Autoregressive Video Modelscited by 189
  36. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  37. 2017Modulating early visual processing by languagecited by 167
  38. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 20; 2024: 37; 2025: 16; 2026: 5.

78 papers.

Sort:

By area:

  1. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  2. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  3. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  4. 2023A Survey on Video Diffusion Modelscited by 203
  5. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Modelscited by 172
  6. 2023Video Probabilistic Diffusion Models in Projected Latent Spacecited by 171
  7. 2024FIFO-Diffusion: Generating Infinite Videos from Text without Trainingcited by 134
  8. 2023Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolutioncited by 131
  9. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
  10. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Taskcited by 91
  11. 2023Animate-A-Story: Storytelling with Retrieval-Augmented Video Generationcited by 89
  12. 2023DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Modelscited by 85
38 more
  1. 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 79
  2. 2023VillanDiffusion: A Unified Backdoor Attack Framework for Diffusion Modelscited by 68
  3. 2024Evaluation of Text-to-Video Generation Models: A Dynamics Perspectivecited by 66
  4. 2024Scaling Diffusion Transformers to 16 Billion Parameterscited by 65
  5. 2023DiffCast: A Unified Framework via Residual Diffusion for Precipitation Nowcastingcited by 57
  6. 2024DiG: Scalable and Efficient Diffusion Models with Gated Linear Attentioncited by 56
  7. 2025AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusioncited by 53
  8. 2023AutoDecoding Latent 3D Diffusion Modelscited by 50
  9. 2024Scalable Diffusion Models with State Space Backbonecited by 45
  10. 2024I2VControl-Camera: Precise Video Camera Control with Adjustable Motion Strengthcited by 38
  11. 2023Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMscited by 35
  12. 2024DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Modelscited by 33
  13. 2023DreamInpainter: Text-Guided Subject-Driven Image Inpainting with Diffusion Modelscited by 30
  14. 2023CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generationcited by 26
  15. 2024Dimba: Transformer-Mamba Diffusion Modelscited by 26
  16. 2024Matten: Video Generation with Mamba-Attentioncited by 24
  17. 2024Hierarchical Patch Diffusion Models for High-Resolution Video Generationcited by 24
  18. 2024Bigger is not Always Better: Scaling Properties of Latent Diffusion Modelscited by 21
  19. 2024TextToon: Real-Time Text Toonify Head Avatar from Single Videocited by 21
  20. 2024Distribution-Aware Data Expansion with Diffusion Modelscited by 20
  21. 2024I2VControl: Disentangled and Unified Video Motion Synthesis Controlcited by 19
  22. 2024Fashion-VDM: Video Diffusion Model for Virtual Try-Oncited by 16
  23. 2025X-Streamer: Unified Human World Modeling with Audiovisual Interactioncited by 16
  24. 2024Automated data processing and feature engineering for deep learning and big data applications: a surveycited by 14
  25. 2024Exploring AIGC Video Quality: A Focus on Visual Harmony, Video-Text Consistency and Domain Distribution Gapcited by 13
  26. 2024CoSIGN: Few-Step Guidance of ConSIstency Model to Solve General INverse Problemscited by 12
  27. 2025MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillationcited by 12
  28. 2024Frame by Familiar Frame: Understanding Replication in Video Diffusion Modelscited by 11
  29. 2024BinaryDM: Accurate Weight Binarization for Efficient Diffusion Modelscited by 11
  30. 2024E-Motion: Future Motion Simulation via Event Sequence Diffusioncited by 11
  31. 2024BiDM: Pushing the Limit of Quantization for Diffusion Modelscited by 11
  32. 2025MotionAgent: Fine-grained Controllable Video Generation via Motion Field Agentcited by 11
  33. 2024Pre-trained Visual Dynamics Representations for Efficient Policy Learningcited by 8
  34. 2025Anomaly Detection and Generation with Diffusion Models: A Surveycited by 8
  35. 2023MCM: Multi-condition Motion Synthesis Framework for Multi-scenariocited by 7
  36. 2024ShowHowTo: Generating Scene-Conditioned Step-by-Step Visual Instructionscited by 7
  37. 2023GLOBER: Coherent Non-autoregressive Video Generation via GLOBal Guided Video DecodERcited by 6
  38. 2023SDDPM: Speckle Denoising Diffusion Probabilistic Modelscited by 6

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applications
  2. 2024SF-V: Single Forward Video Generation Model
  3. 2024Accelerating Video Diffusion Models via Distribution Matching
  4. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation
  5. 2023GD-VDM: Generated Depth for better Diffusion-based Video Generation
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
  7. 2022SinFusion: Training Diffusion Models on a Single Image or Video
  8. 2022Imagen Video: High Definition Video Generation with Diffusion Models