Anchoring and Rescaling Attention for Semantically Coherent Inbetweening
Not on the map. Why?
Abstract (arXiv)
Generative inbetweening (GI) seeks to synthesize realistic intermediate frames between the first and last keyframes beyond mere interpolation. As sequences become sparser and motions larger, previous GI models struggle with inconsistent frames with unstable pacing and semantic misalignment. Since GI involves fixed endpoints and numerous plausible paths, this task requires additional guidance gained from the keyframes and text to specify the intended path. Thus, we give semantic and temporal guidance from the keyframes and text onto each intermediate frame through Keyframe-anchored Attention Bias. We also better enforce frame consistency with Rescaled Temporal RoPE, which allows self-attention to attend to keyframes more faithfully. TGI-Bench, the first benchmark specifically designed for text-conditioned GI evaluation, enables challenge-targeted evaluation to analyze GI models. Without additional training, our method achieves state-of-the-art frame consistency, semantic fidelity, and pace stability for both short and long sequences across diverse challenges.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 38 of its 42 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
38 of 42 references.
References · 38
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
26 more
- 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
- 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
- 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
- 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
- 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 280
- 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 255
- 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 247
- 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
- 2024Autoregressive Video Generation without Vector Quantizationcited by 186
- 2022FILM: Frame Interpolation for Large Motioncited by 153
- 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 151
- 2024Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editingcited by 135
- 2023Scaling Laws of RoPE-based Extrapolationcited by 125
- 2025Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understandingcited by 81
- 2025VideoRoPE: What Makes for Good Video Rotary Position Embedding?cited by 65
- 2020Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolutioncited by 63
- 2024Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolationcited by 44
- 2024Mind the Time: Temporally-Controlled Multi-Event Video Generationcited by 40
- 2024Generative Inbetweening through Frame-wise Conditions-Driven Video Generationcited by 24
- 2024ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Samplercited by 21
- 2025DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusioncited by 14
- 2024LAVIB: A Large-scale Video Interpolation Benchmarkcited by 11
- 2023Video Frame Interpolation with Stereo Event and Intensity Cameracited by 10
- 2025AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolationcited by 9
- 2025WAVE: Warp-Based View Guidance for Consistent Novel View Synthesis Using a Single Imagecited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.