# I2V-Adapter: A General Image-to-Video Adapter for Diffusion Models

arXiv: https://arxiv.org/abs/2312.16693 · HTML: https://mlsanity.com/paper/2312.16693/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 24 of its 54 references that are in the map.
- First posted on arXiv 27 Dec 2023.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 24 of 24)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [VideoCrafter1: Open Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2310.19512/) (2023)
- [DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors](https://mlsanity.com/paper/2310.12190/) (2023)
- [Latent Video Diffusion Models for High-Fidelity Long Video Generation](https://mlsanity.com/paper/2211.13221/) (2022)
- [I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models](https://mlsanity.com/paper/2311.04145/) (2023)
- [LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models](https://mlsanity.com/paper/2309.15103/) (2023)
- [EvalCrafter: Benchmarking and Evaluating Large Video Generation Models](https://mlsanity.com/paper/2310.11440/) (2023)
- [MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model](https://mlsanity.com/paper/2311.16498/) (2023)
- [Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning](https://mlsanity.com/paper/2311.10709/) (2023)
- [SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction](https://mlsanity.com/paper/2310.20700/) (2023)
- [SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models](https://mlsanity.com/paper/2311.16933/) (2023)
- [Make Pixels Dance: High-Dynamic Video Generation](https://mlsanity.com/paper/2311.10982/) (2023)
- [VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation](https://mlsanity.com/paper/2309.00398/) (2023)
- [LAMP: Learn A Motion Pattern for Few-Shot-Based Video Generation](https://mlsanity.com/paper/2310.10769/) (2023)
- [Investigating Prompt Engineering in Diffusion Models](https://mlsanity.com/paper/2211.15462/) (2022)
- [LivePhoto: Real Image Animation with Text-guided Motion Control](https://mlsanity.com/paper/2312.02928/) (2023)

## Built on by (most cited first, 30 of 70)

- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [Controllable Video Generation: A Survey](https://mlsanity.com/paper/2507.16869/) (2025)
- [VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Control](https://mlsanity.com/paper/2503.05639/) (2025)
- [VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2411.13503/) (2024)
- [Parameter-Efficient Fine-Tuning for Foundation Models](https://mlsanity.com/paper/2501.13787/) (2025)
- [VideoWorld: Exploring Knowledge Learning from Unlabeled Videos](https://mlsanity.com/paper/2501.09781/) (2025)
- [ATI: Any Trajectory Instruction for Controllable Video Generation](https://mlsanity.com/paper/2505.22944/) (2025)
- [VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Control](https://mlsanity.com/paper/2501.01427/) (2025)
- [GenFusion: Closing the Loop between Reconstruction and Generation via Videos](https://mlsanity.com/paper/2503.21219/) (2025)
- [I2VControl-Camera: Precise Video Camera Control with Adjustable Motion Strength](https://mlsanity.com/paper/2411.06525/) (2024)
- [Rethinking Video Generation Model for the Embodied World](https://mlsanity.com/paper/2601.15282/) (2026)
- [Exploring the Evolution of Physics Cognition in Video Generation: A Survey](https://mlsanity.com/paper/2503.21765/) (2025)
- [MotionPro: A Precise Motion Controller for Image-to-Video Generation](https://mlsanity.com/paper/2505.20287/) (2025)
- [StableV2V: Stablizing Shape Consistency in Video-to-Video Editing](https://mlsanity.com/paper/2411.11045/) (2024)
- [VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generation](https://mlsanity.com/paper/2502.07531/) (2025)
- [Human Motion Video Generation: A Survey](https://mlsanity.com/paper/2509.03883/) (2025)
- [Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmark](https://mlsanity.com/paper/2402.02242/) (2024)
- [Generative Physical AI in Vision: A Survey](https://mlsanity.com/paper/2501.10928/) (2025)
- [MAGREF: Masked Guidance for Any-Reference Video Generation with Subject Disentanglement](https://mlsanity.com/paper/2505.23742/) (2025)
- [DiffSim: Taming Diffusion Models for Evaluating Visual Similarity](https://mlsanity.com/paper/2412.14580/) (2024)
- [Multi-Reward as Condition for Instruction-based Image Editing](https://mlsanity.com/paper/2411.04713/) (2024)
- [I2VControl: Disentangled and Unified Video Motion Synthesis Control](https://mlsanity.com/paper/2411.17765/) (2024)
- [CINEMA: Coherent Multi-Subject Video Generation via MLLM-Based Guidance](https://mlsanity.com/paper/2503.10391/) (2025)
- [A Survey of Sustainability in Large Language Models: Applications, Economics, and Challenges](https://mlsanity.com/paper/2412.04782/) (2024)
- [GeoVideo: Introducing Geometric Regularization into Video Generation Model](https://mlsanity.com/paper/2512.03453/) (2025)
- [DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Poses](https://mlsanity.com/paper/2412.00397/) (2024)
- [MagicTryOn: Harnessing Diffusion Transformer for Garment-Preserving Video Virtual Try-on](https://mlsanity.com/paper/2505.21325/) (2025)
- [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025)
- [GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting](https://mlsanity.com/paper/2410.05259/) (2024)
- [InterDyn: Controllable Interactive Dynamics with Video Diffusion Models](https://mlsanity.com/paper/2412.11785/) (2024)

## Abstract (arXiv)

Text-guided image-to-video (I2V) generation aims to generate a coherent video that preserves the identity of the input image and semantically aligns with the input prompt. Existing methods typically augment pretrained text-to-video (T2V) models by either concatenating the image with noised video frames channel-wise before being fed into the model or injecting the image embedding produced by pretrained image encoders in cross-attention modules. However, the former approach often necessitates altering the fundamental weights of pretrained T2V models, thus restricting the model's compatibility within the open-source communities and disrupting the model's prior knowledge. Meanwhile, the latter typically fails to preserve the identity of the input image. We present I2V-Adapter to overcome such limitations. I2V-Adapter adeptly propagates the unnoised input image to subsequent noised frames through a cross-frame attention mechanism, maintaining the identity of the input image without any changes to the pretrained T2V model. Notably, I2V-Adapter only introduces a few trainable parameters, significantly alleviating the training cost and also ensures compatibility with existing community-driven personalized models and control tools. Moreover, we propose a novel Frame Similarity Prior to balance the motion amplitude and the stability of generated videos through two adjustable control coefficients. Our experimental results demonstrate that I2V-Adapter is capable of producing high-quality videos. This performance, coupled with its agility and adaptability, represents a substantial advancement in the field of I2V, particularly for personalized and controllable applications.