mlsanityData to 14 Oct 2026

Improving Motion in Image-to-Video Models via Adaptive Low-Pass Guidance

arXiv 2506.08456 ↗First posted 10 Jun 2025cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Recent text-to-video (T2V) models have demonstrated strong capabilities in producing high-quality, dynamic videos. To improve the visual controllability, recent works have considered fine-tuning pre-trained T2V models to support image-to-video (I2V) generation. However, such adaptation frequently suppresses motion dynamics of generated outputs, resulting in more static videos compared to their T2V counterparts. In this work, we analyze this phenomenon and identify that it stems from the premature exposure to high-frequency details in the input image, which biases the sampling process toward a shortcut trajectory that overfits to the static appearance of the reference image. To address this, we propose adaptive low-pass guidance (ALG), a simple training-free fix to the I2V model sampling procedure to generate more dynamic videos without compromising per-frame image quality. Specifically, ALG adaptively modulates the frequency content of the conditioning image by applying a low-pass filter at the early stage of denoising. Extensive experiments show ALG significantly improves the temporal dynamics of generated videos, while preserving or even improving image fidelity and text alignment. For instance, on the VBench test suite, ALG achieves a 33% average improvement across models in dynamic degree while maintaining the original video quality. For additional visualizations and source code, see the project page.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 61 of its 67 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

61 of 67 references.

References · 61

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2022Scalable Diffusion Models with Transformerscited by 6,054
  11. 2022Classifier-Free Diffusion Guidancecited by 5,847
  12. 2022Flow Matching for Generative Modelingcited by 5,702
38 more
  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  8. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  9. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  12. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  13. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  14. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  15. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  16. 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
  17. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  18. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  19. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  20. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  21. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  22. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
  23. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  24. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
  25. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  26. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 388
  27. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
  28. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 350
  29. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  30. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 324
  31. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
  32. 2023Photorealistic Video Generation with Diffusion Modelscited by 306
  33. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 298
  34. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  35. 2025MAGI-1: Autoregressive Video Generation at Scalecited by 276
  36. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 211
  37. 2024Autoregressive Video Generation without Vector Quantizationcited by 186
  38. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 181

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.