mlsanityData to 7 Oct 2026

Motion-Aware Generative Frame Interpolation

arXiv 2501.03699 ↗First posted 7 Jan 2025cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Flow-based frame interpolation methods ensure motion stability through estimated intermediate flow but often introduce severe artifacts in complex motion regions. Recent generative approaches, boosted by large-scale pre-trained video generation models, show promise in handling intricate scenes. However, they frequently produce unstable motion and content inconsistencies due to the absence of explicit motion trajectory constraints. To address these challenges, we propose Motion-aware Generative frame interpolation (MoG) that synergizes intermediate flow guidance with generative capacities to enhance interpolation fidelity. Our key insight is to simultaneously enforce motion smoothness through flow constraints while adaptively correcting flow estimation errors through generative refinement. Specifically, we first introduce a dual guidance injection that propagates condition information using intermediate flow at both latent and feature levels, aligning the generated motion with flow-derived motion trajectories. Meanwhile, we implemented two critical designs, encoder-only guidance injection and selective parameter fine-tuning, which enable dynamic artifact correction in the complex motion regions. Extensive experiments on both real-world and animation benchmarks demonstrate that MoG outperforms state-of-the-art methods in terms of video quality and visual fidelity. Our work bridges the gap between flow-based stability and generative flexibility, offering a versatile solution for frame interpolation across diverse scenarios.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 7 Jan 2025.

Builds on

46 of 59 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  12. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
34 more
  1. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  2. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  3. 2017Video Enhancement with Task-Oriented Flowcited by 612
  4. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  5. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 324
  6. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 321
  7. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  8. 2020Softmax Splatting for Video Frame Interpolationcited by 276
  9. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  10. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 245
  11. 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 240
  12. 2019Depth-Aware Video Frame Interpolationcited by 232
  13. 2020Real-Time Intermediate Flow Estimation for Video Frame Interpolationcited by 211
  14. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  15. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  16. 2018Context-aware Synthesis for Video Frame Interpolationcited by 153
  17. 2021Video Frame Interpolation Transformercited by 122
  18. 2022IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolationcited by 107
  19. 2023Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolationcited by 97
  20. 2023LDMVFI: Video Frame Interpolation with Latent Diffusion Modelscited by 88
  21. 2024ToonCrafter: Generative Cartoon Interpolationcited by 68
  22. 2024Video Interpolation with Diffusion Modelscited by 67
  23. 2022Many-to-many Splatting for Efficient Video Frame Interpolationcited by 42
  24. 2024Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolationcited by 42
  25. 2024One Diffusion to Generate Them Allcited by 41
  26. 2024LeviTor: 3D Trajectory Oriented Image-to-Video Synthesiscited by 39
  27. 2024Sparse Global Matching for Video Frame Interpolation with Large Motioncited by 27
  28. 2023BiFormer: Learning Bilateral Motion Estimation via Bilateral Transformer for 4K Video Frame Interpolationcited by 23
  29. 2024Generative Inbetweening through Frame-wise Conditions-Driven Video Generationcited by 23
  30. 2024VFIMamba: Video Frame Interpolation with State Space Modelscited by 22
  31. 2024Explorative Inbetweening of Time and Spacecited by 21
  32. 2022Splatting-based Synthesis for Video Frame Interpolationcited by 20
  33. 2024ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Samplercited by 19
  34. 2024MotionBridge: Dynamic Video Inbetweening with Flexible Controlscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 6.

13 papers.

Sort:

By area: