Image-to-Video Diffusion: From Foundations to Open Frontiers
On the map: Diffusion models → Video generation
Abstract (arXiv)
Diffusion-based textitimage-to-video (I2V) generation has become a central direction in generative models by turning a reference image, with optional conditions, into a temporally coherent video. Compared with broader video generation settings, this task places stricter demands on content consistency, identity preservation, and motion coherence. Although the literature grows rapidly, existing works mostly discuss I2V generation within broader topics and still lack a dedicated taxonomy together with a systematic analysis centered on this field. This work addresses that gap by treating diffusion I2V generation as a standalone subject. It first reviews the task formulation, model architectures, datasets, and evaluation metrics, and then organizes existing methods through a taxonomy based on architecture and training paradigm. It further distills four core designs, namely condition encoding, temporal modeling, noise prior design, and spatial-temporal upsampling, and discusses representative application scenarios together with major open challenges.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 169 of its 190 references that are in the map. First posted on arXiv 17 May 2026.
Builds on
169 of 190 references.
Closest ancestors · 64
Same direction, newest first
- 2026Tele-Omni: a Unified Multimodal Framework for Video Generation and Editing10 Feb
- 2026ReRoPE: Repurposing RoPE for Relative Camera Control8 Feb
- 2026FSVideo: Fast Speed Video Diffusion Model in a Highly-Compressed Latent Space2 Feb
- 2026Moaw: Unleashing Motion Awareness for Video Diffusion Models19 Jan
- 2026Efficient Camera-Controlled Video Generation of Static Scenes via Sparse Diffusion and 3D Rendering14 Jan
- 2026PhyRPR: Training-Free Physics-Constrained Video Generation14 Jan
- 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer29 Sep
- 2025Versatile Transition Generation with Image-to-Video Diffusion3 Aug
- 2025Seedance 1.0: Exploring the Boundaries of Video Generation Models10 Jun
- 2025OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulation2 Jun
- 2025Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation24 May
- 2025LMP: Leveraging Motion Prior in Zero-Shot Video Generation with Diffusion Transformer20 May
38 more
- 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applications22 Apr
- 2025SkyReels-V2: Infinite-length Film Generative Model17 Apr
- 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models17 Apr
- 2025UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformer15 Apr
- 2025Multi-identity Human Image Animation with Structural Video Diffusion5 Apr
- 2025VACE: All-in-One Video Creation and Editing10 Mar
- 2025Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance10 Feb
- 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generation7 Feb
- 2025UniCP: A Unified Caching and Pruning Framework for Efficient Video Generation6 Feb
- 2025MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generation6 Feb
- 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity3 Feb
- 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Control7 Jan
- 2024LTX-Video: Realtime Video Latent Diffusion30 Dec
- 2024VideoMaker: Zero-shot Customized Video Generation with the Inherent Force of Video Diffusion Models27 Dec
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Models10 Dec
- 2024StableAnimator: High-Quality Identity-Preserving Human Image Animation26 Nov
- 2024MarDini: Masked Autoregressive Diffusion for Video Generation at Scale26 Oct
- 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Model21 Oct
- 2024FrameBridge: Improving Image-to-Video Generation with Bridge Models20 Oct
- 2024TVG: A Training-free Transition Video Generation Method with Diffusion Models24 Aug
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generation31 Jul
- 2024Diffusion Model-Based Video Editing: A Survey26 Jun
- 2024AID: Adapting Image2Video Diffusion Models for Instruction-guided Video Prediction10 Jun
- 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation4 Jun
- 2024I4VGen: Image as Free Stepping Stone for Text-to-Video Generation4 Jun
- 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation3 Jun
- 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model30 May
- 2024EasyAnimate: High-Performance Video Generation Framework with Hybrid Windows Attention and Reward Backpropagation29 May
- 2024Human4DiT: 360-degree Human Video Generation with 4D Diffusion Transformer27 May
- 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models7 May
- 2024TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models25 Apr
- 2024LoopAnimate: Loopable Salient Object Animation14 Apr
- 2024TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Models25 Mar
- 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance21 Mar
- 2024Lumiere: A Space-Time Diffusion Model for Video Generation23 Jan
- 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
- 2024Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditions3 Jan
50 newest of 64 All 64 (JSON)
Other references · 105
- 2017Attention Is All You Needcited by 59,932
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
38 more
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
- 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
- 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
- 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
- 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
- 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
- 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
- 2025MAGI-1: Autoregressive Video Generation at Scalecited by 262
- 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
- 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 246
- 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
- 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
- 2018A Note on the Inception Scorecited by 225
- 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
- 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 203
- 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 198
- 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
- 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
- 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 191
- 2023NeRSemble: Multi-view Radiance Field Reconstruction of Human Headscited by 190
- 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 174
- 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 169
- 2018Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separationcited by 160
- 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
- 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
- 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
- 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
- 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 121
- 2024Boximator: Generating Rich and Controllable Motions for Video Synthesiscited by 107
- 2023CelebV-Text: A Large-Scale Facial Text-Video Datasetcited by 103
- 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependencycited by 102
50 most cited of 105 All 105 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2022Video Diffusion Models
- 2026Efficient Video Diffusion Models: Advancements and Challenges
- 2024Video Diffusion Models: A Survey
- 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation
- 2024S2DM: Sector-Shaped Diffusion Models for Video Generation
- 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models
- 2024Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Model
- 2026UniVid: Pyramid Diffusion Model for High Quality Video Generation