mlsanityData to 7 Oct 2026

Image-to-Video Diffusion: From Foundations to Open Frontiers

arXiv 2605.17248 ↗First posted 17 May 2026cs.CVcited by 1

On the map: Diffusion models → Video generation

Abstract (arXiv)

Diffusion-based textitimage-to-video (I2V) generation has become a central direction in generative models by turning a reference image, with optional conditions, into a temporally coherent video. Compared with broader video generation settings, this task places stricter demands on content consistency, identity preservation, and motion coherence. Although the literature grows rapidly, existing works mostly discuss I2V generation within broader topics and still lack a dedicated taxonomy together with a systematic analysis centered on this field. This work addresses that gap by treating diffusion I2V generation as a standalone subject. It first reviews the task formulation, model architectures, datasets, and evaluation metrics, and then organizes existing methods through a taxonomy based on architecture and training paradigm. It further distills four core designs, namely condition encoding, temporal modeling, noise prior design, and spatial-temporal upsampling, and discusses representative application scenarios together with major open challenges.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 169 of its 190 references that are in the map. First posted on arXiv 17 May 2026.

Builds on

169 of 190 references.

Closest ancestors · 64

Same direction, newest first

  1. 2026Tele-Omni: a Unified Multimodal Framework for Video Generation and Editing10 Feb
  2. 2026ReRoPE: Repurposing RoPE for Relative Camera Control8 Feb
  3. 2026FSVideo: Fast Speed Video Diffusion Model in a Highly-Compressed Latent Space2 Feb
  4. 2026Moaw: Unleashing Motion Awareness for Video Diffusion Models19 Jan
  5. 2026Efficient Camera-Controlled Video Generation of Static Scenes via Sparse Diffusion and 3D Rendering14 Jan
  6. 2026PhyRPR: Training-Free Physics-Constrained Video Generation14 Jan
  7. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer29 Sep
  8. 2025Versatile Transition Generation with Image-to-Video Diffusion3 Aug
  9. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Models10 Jun
  10. 2025OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulation2 Jun
  11. 2025Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation24 May
  12. 2025LMP: Leveraging Motion Prior in Zero-Shot Video Generation with Diffusion Transformer20 May
38 more
  1. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applications22 Apr
  2. 2025SkyReels-V2: Infinite-length Film Generative Model17 Apr
  3. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models17 Apr
  4. 2025UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformer15 Apr
  5. 2025Multi-identity Human Image Animation with Structural Video Diffusion5 Apr
  6. 2025VACE: All-in-One Video Creation and Editing10 Mar
  7. 2025Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance10 Feb
  8. 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generation7 Feb
  9. 2025UniCP: A Unified Caching and Pruning Framework for Efficient Video Generation6 Feb
  10. 2025MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generation6 Feb
  11. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity3 Feb
  12. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Control7 Jan
  13. 2024LTX-Video: Realtime Video Latent Diffusion30 Dec
  14. 2024VideoMaker: Zero-shot Customized Video Generation with the Inherent Force of Video Diffusion Models27 Dec
  15. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Models10 Dec
  16. 2024StableAnimator: High-Quality Identity-Preserving Human Image Animation26 Nov
  17. 2024MarDini: Masked Autoregressive Diffusion for Video Generation at Scale26 Oct
  18. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Model21 Oct
  19. 2024FrameBridge: Improving Image-to-Video Generation with Bridge Models20 Oct
  20. 2024TVG: A Training-free Transition Video Generation Method with Diffusion Models24 Aug
  21. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  22. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generation31 Jul
  23. 2024Diffusion Model-Based Video Editing: A Survey26 Jun
  24. 2024AID: Adapting Image2Video Diffusion Models for Instruction-guided Video Prediction10 Jun
  25. 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation4 Jun
  26. 2024I4VGen: Image as Free Stepping Stone for Text-to-Video Generation4 Jun
  27. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation3 Jun
  28. 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model30 May
  29. 2024EasyAnimate: High-Performance Video Generation Framework with Hybrid Windows Attention and Reward Backpropagation29 May
  30. 2024Human4DiT: 360-degree Human Video Generation with 4D Diffusion Transformer27 May
  31. 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models7 May
  32. 2024TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models25 Apr
  33. 2024LoopAnimate: Loopable Salient Object Animation14 Apr
  34. 2024TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Models25 Mar
  35. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance21 Mar
  36. 2024Lumiere: A Space-Time Diffusion Model for Video Generation23 Jan
  37. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
  38. 2024Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditions3 Jan

50 newest of 64 All 64 (JSON)

Other references · 105

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  10. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  11. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  12. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
38 more
  1. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  2. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  3. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  4. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  5. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  6. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  7. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  8. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  9. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  10. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  11. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  12. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
  13. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
  14. 2025MAGI-1: Autoregressive Video Generation at Scalecited by 262
  15. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  16. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 246
  17. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
  18. 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
  19. 2018A Note on the Inception Scorecited by 225
  20. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
  21. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 203
  22. 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 198
  23. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  24. 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
  25. 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 191
  26. 2023NeRSemble: Multi-view Radiance Field Reconstruction of Human Headscited by 190
  27. 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 174
  28. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 169
  29. 2018Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separationcited by 160
  30. 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
  31. 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
  32. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  33. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  34. 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
  35. 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 121
  36. 2024Boximator: Generating Rich and Controllable Motions for Video Synthesiscited by 107
  37. 2023CelebV-Text: A Large-Scale Facial Text-Video Datasetcited by 103
  38. 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependencycited by 102

50 most cited of 105 All 105 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2022Video Diffusion Models
  2. 2026Efficient Video Diffusion Models: Advancements and Challenges
  3. 2024Video Diffusion Models: A Survey
  4. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation
  5. 2024S2DM: Sector-Shaped Diffusion Models for Video Generation
  6. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models
  7. 2024Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Model
  8. 2026UniVid: Pyramid Diffusion Model for High Quality Video Generation