mlsanityData to 7 Oct 2026

I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models

arXiv 2311.04145 ↗First posted 7 Nov 2023cs.CVcited by 343

On the map: Diffusion models → Video generation

Abstract (arXiv)

Video synthesis has recently made remarkable strides benefiting from the rapid development of diffusion models. However, it still encounters challenges in terms of semantic accuracy, clarity and spatio-temporal continuity. They primarily arise from the scarcity of well-aligned text-video data and the complex inherent structure of videos, making it difficult for the model to simultaneously ensure semantic and qualitative excellence. In this report, we propose a cascaded I2VGen-XL approach that enhances model performance by decoupling these two factors and ensures the alignment of the input data by utilizing static images as a form of crucial guidance. I2VGen-XL consists of two stages: i) the base stage guarantees coherent semantics and preserves content from input images by using two hierarchical encoders, and ii) the refinement stage enhances the video's details by incorporating an additional brief text and improves the resolution to 1280×720. To improve the diversity, we collect around 35 million single-shot text-video pairs and 6 billion text-image pairs to optimize the model. By this means, I2VGen-XL can simultaneously enhance the semantic accuracy, continuity of details and clarity of generated videos. Through extensive experiments, we have investigated the underlying principles of I2VGen-XL and compared it with current top methods, which can demonstrate its effectiveness on diverse data. The source code and models will be publicly available at urlhttps://i2vgen-xl.github.io.

Key facts. Cited by 343 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 54 of its 60 references that are in the map. First posted on arXiv 7 Nov 2023.

Builds on

54 of 60 references.

Closest ancestors · 15

Same direction, newest first

  1. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generation30 Oct
  2. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors18 Oct
  3. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models26 Sep
  4. 2023ModelScope Text-to-Video Technical Report12 Aug
  5. 2023VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNet26 Jul
  6. 2023Video ControlNet: Towards Temporally Consistent Synthetic-to-Real Video Translation Using Conditional Image Diffusion Models30 May
  7. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  8. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation15 Mar
  9. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
  10. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models20 Nov
  11. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  12. 2022Diffusion Models for Video Prediction and Infilling15 Jun
3 more
  1. 2022Flexible Diffusion Modeling of Long Videos23 May
  2. 2022Video Diffusion Models7 Apr
  3. 2022Diffusion Probabilistic Modeling for Video Generation16 Mar

Other references · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2022Generative Adversarial Networkscited by 5,603
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
27 more
  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  5. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  7. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  8. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  9. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  10. 2021Variational Diffusion Modelscited by 1,145
  11. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  12. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  13. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  14. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  15. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  16. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  17. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  18. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
  19. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  20. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Qualitycited by 181
  21. 2022SINE: SINgle Image Editing with Text-to-Image Diffusion Modelscited by 159
  22. 2023Make-Your-Video: Customized Video Generation Using Textual and Structural Guidancecited by 129
  23. 2022gDDIM: Generalized denoising diffusion implicit modelscited by 125
  24. 2022Riemannian Diffusion Modelscited by 120
  25. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  26. 2022Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoderscited by 42
  27. 2018Learning to Forecast and Refine Residual Motion for Image-to-Video Generationcited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 132; 2025: 146; 2026: 49.

343 papers.

Sort:

By area:

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  3. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  4. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  5. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  6. 2025VACE: All-in-One Video Creation and Editingcited by 467
  7. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  8. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  9. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  10. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Textcited by 242
  11. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  12. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
38 more
  1. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
  2. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  3. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 186
  4. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
  5. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 172
  6. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 169
  7. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 161
  8. 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
  9. 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 151
  10. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  11. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
  12. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  13. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  14. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 115
  15. 2025Long Context Tuning for Video Generationcited by 110
  16. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
  17. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 101
  18. 2024Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Modelscited by 99
  19. 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 91
  20. 2023VideoLCM: Video Latent Consistency Modelcited by 90
  21. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 88
  22. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedbackcited by 85
  23. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
  24. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 84
  25. 2025HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generationcited by 84
  26. 2024Pandora: Towards General World Model with Natural Language Actions and Video Statescited by 83
  27. 2025Automated Movie Generation via Multi-Agent CoT Planningcited by 82
  28. 2024CustomVideo: Customizing Text-to-Video Generation with Multiple Subjectscited by 76
  29. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  30. 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
  31. 2023I2V-Adapter: A General Image-to-Video Adapter for Diffusion Modelscited by 70
  32. 2024Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharingcited by 70
  33. 2024ToonCrafter: Generative Cartoon Interpolationcited by 68
  34. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  35. 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuningcited by 67
  36. 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 65
  37. 2024Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Modelcited by 64
  38. 2024VEnhancer: Generative Space-Time Enhancement for Video Generationcited by 64

50 most cited of 343 All 343 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation
  2. 2025CascadeV: An Implementation of Wurstchen Architecture for Video Generation
  3. 2025UI2V-Bench: An Understanding-based Image-to-video Generation Benchmark
  4. 2024xGen-VideoSyn-1: High-fidelity Text-to-Video Synthesis with Compressed Representations
  5. 2025SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolution
  6. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
  7. 2024S2DM: Sector-Shaped Diffusion Models for Video Generation
  8. 2023BIVDiff: A Training-Free Framework for General-Purpose Video Synthesis via Bridging Image and Video Diffusion Models