AlignVid: Training-Free Attention Scaling for Semantic Fidelity in Text-Guided Image-to-Video Generation
Not on the map. Why?
Abstract (arXiv)
Text-guided image-to-video generation has made substantial progress, yet it still struggles to execute text-specified edits that require substantial changes to a reference image (textite.g., object addition, removal, or modification). Empirically, our analysis reveals that this stems from visual dominance, where the reference image causes severe attention dispersion, inhibiting the model's ability to incorporate new semantic information. To address this, we propose AlignVid, a training-free intervention that re-calibrates the model's internal attention distribution. Drawing on an energy-based perspective of attention, AlignVid employs Attention Scaling Modulation (ASM) to reduce attention entropy and concentrate focus on semantic tokens, alongside Guidance Scheduling (GS) to maintain generation stability. To rigorously assess this capability, we present OmitI2V, a comprehensive benchmark for evaluating prompt adherence across object modification, addition, and deletion. Extensive experiments demonstrate that AlignVid effectively enhances semantic fidelity with negligible computational overhead. Code and the OmitI2V benchmark are available at https://github.com/LAW1223/AlignVid.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 37 of its 43 references that are in the map. First posted on arXiv 1 Dec 2025.
Builds on
37 of 43 references.
References · 37
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2024GPT-4o System Cardcited by 4,467
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
25 more
- 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
- 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
- 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
- 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
- 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
- 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
- 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
- 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 231
- 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
- 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 218
- 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
- 2024Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editingcited by 134
- 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
- 2024DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generationcited by 73
- 2024Zero-shot Image Editing with Reference Imitationcited by 64
- 2023MagicStick: Controllable Video Editing via Control Handle Transformationscited by 62
- 2024Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attentioncited by 55
- 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editingcited by 50
- 2024TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generationcited by 42
- 2025MotionPro: A Precise Motion Controller for Image-to-Video Generationcited by 29
- 2024AIGCBench: Comprehensive Evaluation of Image-to-Video Content Generated by AIcited by 27
- 2025LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to-Text Interpretationcited by 10
- 2023RealCraft: Attention Control as A Tool for Zero-Shot Consistent Video Editingcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.