mlsanityData to 7 Oct 2026

ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation

arXiv 2402.04324 ↗First posted 6 Feb 2024cs.CVcited by 130

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Image-to-video (I2V) generation aims to use the initial frame (alongside a text prompt) to create a video sequence. A grand challenge in I2V generation is to maintain visual consistency throughout the video: existing methods often struggle to preserve the integrity of the subject, background, and style from the first frame, as well as ensure a fluid and logical progression within the video narrative. To mitigate these issues, we propose ConsistI2V, a diffusion-based method to enhance visual consistency for I2V generation. Specifically, we introduce (1) spatiotemporal attention over the first frame to maintain spatial and motion consistency, (2) noise initialization from the low-frequency band of the first frame to enhance layout consistency. These two approaches enable ConsistI2V to generate highly consistent videos. We also extend the proposed approaches to show their potential to improve consistency in auto-regressive long video generation and camera motion control. To verify the effectiveness of our method, we propose I2V-Bench, a comprehensive evaluation benchmark for I2V generation. Our automatic and human evaluation results demonstrate the superiority of ConsistI2V over existing methods.

Key facts. Cited by 130 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 62 of its 73 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

62 of 73 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Learning Transferable Visual Models From Natural Language Supervision26 Feb
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 60

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  11. 2016Improved Techniques for Training GANscited by 2,938
  12. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
38 more
  1. 2022Video Diffusion Modelscited by 2,280
  2. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  3. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  4. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  6. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  7. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  8. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  9. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  10. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  11. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  12. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  13. 2023ModelScope Text-to-Video Technical Reportcited by 651
  14. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  15. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  16. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  17. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  18. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  19. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  20. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  21. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
  22. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 324
  23. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  24. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  25. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  26. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  27. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  28. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  29. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  30. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 231
  31. 2023Dreamix: Video Diffusion Models are General Video Editorscited by 189
  32. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  33. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 173
  34. 2023FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editingcited by 165
  35. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162
  36. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 153
  37. 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
  38. 2023Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generationcited by 137

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 44; 2025: 56; 2026: 30.

130 papers.

Sort:

By area:

  1. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  2. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Textcited by 242
  3. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  4. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  5. 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 151
  6. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 110
  7. 2025Controllable Video Generation: A Surveycited by 93
  8. 2024DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmarkcited by 85
  9. 2024Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharingcited by 70
  10. 2025EnerVerse: Envisioning Embodied Future Space for Robotics Manipulationcited by 59
  11. 2024Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generationcited by 54
  12. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generationcited by 53
38 more
  1. 2025HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narrativescited by 51
  2. 2024MarDini: Masked Autoregressive Diffusion for Video Generation at Scalecited by 49
  3. 2024DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusioncited by 46
  4. 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Modelscited by 45
  5. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  6. 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 42
  7. 2025VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Controlcited by 42
  8. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  9. 2024ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Modelscited by 33
  10. 2025Judge Anything: MLLM as a Judge Across Any Modalitycited by 33
  11. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  12. 2024Video Diffusion Models: A Surveycited by 28
  13. 2024Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Modelcited by 27
  14. 2024STIV: Scalable Text and Image Conditioned Video Generationcited by 24
  15. 2025Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspectivecited by 22
  16. 2024ResMaster: Mastering High-Resolution Image Generation via Structural and Fine-Grained Guidancecited by 21
  17. 2024InfinityDrive: Breaking Time Limits in Driving World Modelscited by 21
  18. 2025FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generationcited by 21
  19. 2025Through-The-Mask: Mask-based Motion Trajectories for Image-to-Video Generationcited by 20
  20. 2025Dynamic Concepts Personalization from Single Videoscited by 20
  21. 2025Error Analyses of Auto-Regressive Video Diffusion Models: A Unified Frameworkcited by 20
  22. 2025QVGen: Pushing the Limit of Quantized Video Generative Modelscited by 20
  23. 2024FrameBridge: Improving Image-to-Video Generation with Bridge Modelscited by 19
  24. 2025EgoExo-Gen: Ego-centric Video Prediction by Watching Exo-centric Videoscited by 19
  25. 2024Owl-1: Omni World Model for Consistent Long Video Generationcited by 17
  26. 2024MotionStone: Decoupled Motion Intensity Modulation with Diffusion Transformer for Image-to-Video Generationcited by 16
  27. 2024FancyVideo: Towards Dynamic and Consistent Video Generation via Cross-frame Textual Guidancecited by 15
  28. 2025VideoMAR: Autoregressive Video Generatio with Continuous Tokenscited by 15
  29. 2024Cinemo: Consistent and Controllable Image Animation with Motion Diffusion Modelscited by 14
  30. 2025DVD-Quant: Data-free Video Diffusion Transformers Quantizationcited by 14
  31. 2025GeoVideo: Introducing Geometric Regularization into Video Generation Modelcited by 14
  32. 2026Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoningcited by 13
  33. 2024I4VGen: Image as Free Stepping Stone for Text-to-Video Generationcited by 12
  34. 2024EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolationcited by 12
  35. 2025Extrapolating and Decoupling Image-to-Video Generation Models: Motion Modeling is Easier Than You Thinkcited by 12
  36. 2025ScalingNoise: Scaling Inference-Time Search for Generating Infinite Videoscited by 12
  37. 2025MotionAgent: Fine-grained Controllable Video Generation via Motion Field Agentcited by 11
  38. 2025Uniform Discrete Diffusion with Metric Path for Video Generationcited by 11

50 most cited of 130 All 130 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025UI2V-Bench: An Understanding-based Image-to-video Generation Benchmark
  2. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Model
  3. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling
  4. 2025View-Consistent Diffusion Representations for 3D-Consistent Video Generation
  5. 2026GroundShot: Visually Consistent Multi-Shot Long Video Generation via Entity-Grounded Shot Scheduling
  6. 2026DCDM: Divide-and-Conquer Diffusion Models for Consistency-Preserving Video Generation
  7. 2025Training-Free Motion-Guided Video Generation with Enhanced Temporal Consistency Using Motion Consistency Loss
  8. 2025Dual-Expert Consistency Model for Efficient and High-Quality Video Generation