Rebalancing Reference Frame Dominance to Improve Motion in Image-to-Video Models
Not on the map. Why?
Abstract (arXiv)
Image-to-video models often generate videos that remain overly static, compared to text-to-video models. While prior approaches mitigate this issue by weakening or modifying the image-conditioning signal, they often require additional training or sacrifice fidelity to the reference image. In this work, we identify reference-frame dominance as a key mechanism behind motion suppression. We observe that non-reference frames in I2V models allocate excessive self-attention to reference-frame key tokens, causing reference information to be over-propagated across time and suppressing inter-frame dynamics. Based on this finding, we propose DyMoS (Dynamic Motion Slider), a training-free and model-agnostic method that rebalances the attention pathway from generated frames to the reference frame during initial denoising steps. DyMoS leaves both the input image and model weights unchanged and introduces a single scalar parameter for continuous control over motion strength. Experiments across multiple state-of-the-art I2V backbones demonstrate that DyMoS consistently improves motion dynamics while maintaining visual quality and fidelity to the reference image.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 43 of its 51 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
43 of 51 references.
References · 43
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
31 more
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
- 2022Video Diffusion Modelscited by 2,305
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
- 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
- 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 737
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
- 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
- 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 350
- 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
- 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
- 2025MAGI-1: Autoregressive Video Generation at Scalecited by 276
- 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 211
- 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
- 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
- 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 181
- 2025History-Guided Video Diffusioncited by 171
- 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generationcited by 156
- 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 131
- 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 44
- 2025Reangle-A-Video: 4D Video Generation as Video-to-Video Translationcited by 35
- 2024Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Modelcited by 28
- 2024Kinetic Typography Diffusion Modelcited by 9
- 2025Video Color Grading via Look-Up Table Generationcited by 9
- 2026Universal Image Immunization against Diffusion-based Image Editing via Semantic Injectioncited by 7
- 2026Motion Prior Distillation in Time Reversal Sampling for Generative Inbetweeningcited by 6
- 2025FlashI2V: Fourier-Guided Latent Shifting Prevents Conditional Image Leakage in Image-to-Video Generationcited by 4
- 2026MotionCFG: Boosting Motion Dynamics via Stochastic Concept Perturbationcited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.