Motion-Aware Generative Frame Interpolation
Not on the map. Why?
Abstract (arXiv)
Flow-based frame interpolation methods ensure motion stability through estimated intermediate flow but often introduce severe artifacts in complex motion regions. Recent generative approaches, boosted by large-scale pre-trained video generation models, show promise in handling intricate scenes. However, they frequently produce unstable motion and content inconsistencies due to the absence of explicit motion trajectory constraints. To address these challenges, we propose Motion-aware Generative frame interpolation (MoG) that synergizes intermediate flow guidance with generative capacities to enhance interpolation fidelity. Our key insight is to simultaneously enforce motion smoothness through flow constraints while adaptively correcting flow estimation errors through generative refinement. Specifically, we first introduce a dual guidance injection that propagates condition information using intermediate flow at both latent and feature levels, aligning the generated motion with flow-derived motion trajectories. Meanwhile, we implemented two critical designs, encoder-only guidance injection and selective parameter fine-tuning, which enable dynamic artifact correction in the complex motion regions. Extensive experiments on both real-world and animation benchmarks demonstrate that MoG outperforms state-of-the-art methods in terms of video quality and visual fidelity. Our work bridges the gap between flow-based stability and generative flexibility, offering a versatile solution for frame interpolation across diverse scenarios.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 7 Jan 2025.
Builds on
46 of 59 references.
References · 46
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
34 more
- 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
- 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
- 2017Video Enhancement with Task-Oriented Flowcited by 612
- 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
- 2021GMFlow: Learning Optical Flow via Global Matchingcited by 324
- 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 321
- 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
- 2020Softmax Splatting for Video Frame Interpolationcited by 276
- 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
- 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 245
- 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 240
- 2019Depth-Aware Video Frame Interpolationcited by 232
- 2020Real-Time Intermediate Flow Estimation for Video Frame Interpolationcited by 211
- 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
- 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
- 2018Context-aware Synthesis for Video Frame Interpolationcited by 153
- 2021Video Frame Interpolation Transformercited by 122
- 2022IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolationcited by 107
- 2023Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolationcited by 97
- 2023LDMVFI: Video Frame Interpolation with Latent Diffusion Modelscited by 88
- 2024ToonCrafter: Generative Cartoon Interpolationcited by 68
- 2024Video Interpolation with Diffusion Modelscited by 67
- 2022Many-to-many Splatting for Efficient Video Frame Interpolationcited by 42
- 2024Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolationcited by 42
- 2024One Diffusion to Generate Them Allcited by 41
- 2024LeviTor: 3D Trajectory Oriented Image-to-Video Synthesiscited by 39
- 2024Sparse Global Matching for Video Frame Interpolation with Large Motioncited by 27
- 2023BiFormer: Learning Bilateral Motion Estimation via Bilateral Transformer for 4K Video Frame Interpolationcited by 23
- 2024Generative Inbetweening through Frame-wise Conditions-Driven Video Generationcited by 23
- 2024VFIMamba: Video Frame Interpolation with State Space Modelscited by 22
- 2024Explorative Inbetweening of Time and Spacecited by 21
- 2022Splatting-based Synthesis for Video Frame Interpolationcited by 20
- 2024ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Samplercited by 19
- 2024MotionBridge: Dynamic Video Inbetweening with Flexible Controlscited by 10
Built on by
Citing papers per year: 2025: 7; 2026: 6.
13 papers.
Sort:
By area:
- 2025Pack and Force Your Memory: Long-form and Consistent Video Generationcited by 26
- 2025AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolationcited by 8
- 2025FreeRet: MLLMs as Training-Free Retrieverscited by 7
- 2025MotionRAG: Motion Retrieval-Augmented Image-to-Video Generationcited by 7
- 2026VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understandingcited by 6
- 2026Motion Prior Distillation in Time Reversal Sampling for Generative Inbetweeningcited by 5
- 2025EF-VI: Enhancing End-Frame Injection for Video Inbetweeningcited by 3
- 2025Arbitrary Generative Video Interpolationcited by 1
- 2025Time-Correlated Video Bridge Matchingcited by 1
- 2026Imagine How To Change: Explicit Procedure Modeling for Change Captioningcited by 1
- 2026FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generationcited by 0
- 2026Ego-InBetween: Generating Object State Transitions in Ego-Centric Videoscited by 0
1 more
- 2026SNM-VFI: Symmetric Nonlinear Motion-Guided Generative Video Frame Interpolation13 Aug 2026
- 2026VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding16 Jul 2026
- 2026Ego-InBetween: Generating Object State Transitions in Ego-Centric Videos20 Apr 2026
- 2026Imagine How To Change: Explicit Procedure Modeling for Change Captioning6 Mar 2026
- 2026FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generation5 Mar 2026
- 2026Motion Prior Distillation in Time Reversal Sampling for Generative Inbetweening13 Feb 2026
- 2025Time-Correlated Video Bridge Matching14 Oct 2025
- 2025Pack and Force Your Memory: Long-form and Consistent Video Generation2 Oct 2025
- 2025Arbitrary Generative Video Interpolation1 Oct 2025
- 2025MotionRAG: Motion Retrieval-Augmented Image-to-Video Generation30 Sep 2025
- 2025FreeRet: MLLMs as Training-Free Retrievers29 Sep 2025
- 2025AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation1 Jun 2025
1 more
- 2025EF-VI: Enhancing End-Frame Injection for Video Inbetweening27 May 2025
- 2025MotionRAG: Motion Retrieval-Augmented Image-to-Video Generationcited by 7
- 2026Motion Prior Distillation in Time Reversal Sampling for Generative Inbetweeningcited by 5
- 2025Time-Correlated Video Bridge Matchingcited by 1
- 2026FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generationcited by 0
- 2026SNM-VFI: Symmetric Nonlinear Motion-Guided Generative Video Frame Interpolationcited by 0
- 2026SNM-VFI: Symmetric Nonlinear Motion-Guided Generative Video Frame Interpolation13 Aug 2026
- 2026FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generation5 Mar 2026
- 2026Motion Prior Distillation in Time Reversal Sampling for Generative Inbetweening13 Feb 2026
- 2025Time-Correlated Video Bridge Matching14 Oct 2025
- 2025MotionRAG: Motion Retrieval-Augmented Image-to-Video Generation30 Sep 2025
- 2025FreeRet: MLLMs as Training-Free Retrieverscited by 7
- 2025FreeRet: MLLMs as Training-Free Retrievers29 Sep 2025
- 2025Pack and Force Your Memory: Long-form and Consistent Video Generationcited by 26
- 2025AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolationcited by 8
- 2026VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understandingcited by 6
- 2025EF-VI: Enhancing End-Frame Injection for Video Inbetweeningcited by 3
- 2025Arbitrary Generative Video Interpolationcited by 1
- 2026Imagine How To Change: Explicit Procedure Modeling for Change Captioningcited by 1
- 2026Ego-InBetween: Generating Object State Transitions in Ego-Centric Videoscited by 0
- 2026VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding16 Jul 2026
- 2026Ego-InBetween: Generating Object State Transitions in Ego-Centric Videos20 Apr 2026
- 2026Imagine How To Change: Explicit Procedure Modeling for Change Captioning6 Mar 2026
- 2025Pack and Force Your Memory: Long-form and Consistent Video Generation2 Oct 2025
- 2025Arbitrary Generative Video Interpolation1 Oct 2025
- 2025AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation1 Jun 2025
- 2025EF-VI: Enhancing End-Frame Injection for Video Inbetweening27 May 2025