# Sparse Global Matching for Video Frame Interpolation with Large Motion

arXiv: https://arxiv.org/abs/2404.06913 · HTML: https://mlsanity.com/paper/2404.06913/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 41 references that are in the map.
- First posted on arXiv 10 Apr 2024.

## Builds on (most cited first, 30 of 36)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Real-Time Intermediate Flow Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2011.06294/) (2020)
- [DeepStereo: Learning to Predict New Views from the World's Imagery](https://mlsanity.com/paper/1506.06825/) (2015)
- [COTR: Correspondence Transformer for Matching Across Images](https://mlsanity.com/paper/2103.14167/) (2021)
- [AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation](https://mlsanity.com/paper/2304.09790/) (2023)
- [FILM: Frame Interpolation for Large Motion](https://mlsanity.com/paper/2202.04901/) (2022)
- [Quadratic video interpolation](https://mlsanity.com/paper/1911.00627/) (2019)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [XVFI: eXtreme Video Frame Interpolation](https://mlsanity.com/paper/2103.16206/) (2021)
- [Video Frame Interpolation Transformer](https://mlsanity.com/paper/2111.13817/) (2021)
- [BMBC:Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation](https://mlsanity.com/paper/2007.12622/) (2020)
- [IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation](https://mlsanity.com/paper/2205.14620/) (2022)
- [Asymmetric Bilateral Motion Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2108.06815/) (2021)
- [VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution](https://mlsanity.com/paper/2206.04647/) (2022)
- [Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation](https://mlsanity.com/paper/2303.00440/) (2023)
- [Deep Animation Video Interpolation in the Wild](https://mlsanity.com/paper/2104.02495/) (2021)
- [Enhanced Quadratic Video Interpolation](https://mlsanity.com/paper/2009.04642/) (2020)
- [Many-to-many Splatting for Efficient Video Frame Interpolation](https://mlsanity.com/paper/2204.03513/) (2022)
- [A Unified Pyramid Recurrent Network for Video Frame Interpolation](https://mlsanity.com/paper/2211.03456/) (2022)
- [Learning to Synthesize Motion Blur](https://mlsanity.com/paper/1811.11745/) (2018)

## Built on by (most cited first, 28 of 28)

- [AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportation](https://mlsanity.com/paper/2407.04603/) (2024)
- [Generative Inbetweening through Frame-wise Conditions-Driven Video Generation](https://mlsanity.com/paper/2412.11755/) (2024)
- [VFIMamba: Video Frame Interpolation with State Space Models](https://mlsanity.com/paper/2407.02315/) (2024)
- [EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation](https://mlsanity.com/paper/2503.15831/) (2025)
- [PMQ-VE: Progressive Multi-Frame Quantization for Video Enhancement](https://mlsanity.com/paper/2505.12266/) (2025)
- [Motion-Aware Generative Frame Interpolation](https://mlsanity.com/paper/2501.03699/) (2025)
- [AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation](https://mlsanity.com/paper/2506.01061/) (2025)
- [High-Resolution Frame Interpolation with Patch-based Cascaded Diffusion](https://mlsanity.com/paper/2410.11838/) (2024)
- [Hierarchical Flow Diffusion for Efficient Frame Interpolation](https://mlsanity.com/paper/2504.00380/) (2025)
- [Versatile Transition Generation with Image-to-Video Diffusion](https://mlsanity.com/paper/2508.01698/) (2025)
- [Thin-Plate Spline-based Interpolation for Animation Line Inbetweening](https://mlsanity.com/paper/2408.09131/) (2024)
- [EF-VI: Enhancing End-Frame Injection for Video Inbetweening](https://mlsanity.com/paper/2505.21205/) (2025)
- [Towards Holistic Modeling for Video Frame Interpolation with Auto-regressive Diffusion Transformers](https://mlsanity.com/paper/2601.14959/) (2026)
- [VTinker: Guided Flow Upsampling and Texture Mapping for High-Resolution Video Frame Interpolation](https://mlsanity.com/paper/2511.16124/) (2025)
- [MIORe & VAR-MIORe: Benchmarks to Push the Boundaries of Restoration](https://mlsanity.com/paper/2509.06803/) (2025)
- [SPEED: One-Step Pixel Diffusion for High-quality Video Frame Interpolation](https://mlsanity.com/paper/2607.15585/) (2026)
- [LumosFlow: Motion-Guided Long Video Generation](https://mlsanity.com/paper/2506.02497/) (2025)
- [Towards Robust and Generalizable Continuous Space-Time Video Super-Resolution with Events](https://mlsanity.com/paper/2510.03833/) (2025)
- [Edit2Interp: Adapting Image Foundation Models from Spatial Editing to Video Frame Interpolation with Few-Shot Learning](https://mlsanity.com/paper/2603.15003/) (2026)
- [Can Video Diffusion Models Predict Past Frames? Bidirectional Cycle Consistency for Reversible Interpolation](https://mlsanity.com/paper/2604.01700/) (2026)
- [MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction](https://mlsanity.com/paper/2604.01958/) (2026)
- [TVRN: Invertible Neural Networks for Compression-Aware Temporal Video Rescaling](https://mlsanity.com/paper/2605.15579/) (2026)
- [MASS: Motion-Aligned Selective Scan for Refinement in Flow-Based Video Frame Interpolation](https://mlsanity.com/paper/2606.27718/) (2026)
- [T2exture: Sparsely Perturbed Thermal-to-Texture Imaging](https://mlsanity.com/paper/2608.02192/) (2026)
- [Multiple Hypothesis Flow Estimation for Video Frame Interpolation under Matching Ambiguity](https://mlsanity.com/paper/2608.07120/) (2026)
- [Following Motion for Sequential Modeling in Video Frame Interpolation](https://mlsanity.com/paper/2608.22861/) (2026)
- [Accurate Motion Estimation with B'ezier Control Point for Efficient Frame Interpolation](https://mlsanity.com/paper/2609.23408/) (2026)
- [Destination-Conditioned Adaptation with Semantic-Geometric Alignment for Generative Inbetweening](https://mlsanity.com/paper/2610.13187/) (2026)

## Abstract (arXiv)

Large motion poses a critical challenge in Video Frame Interpolation (VFI) task. Existing methods are often constrained by limited receptive fields, resulting in sub-optimal performance when handling scenarios with large motion. In this paper, we introduce a new pipeline for VFI, which can effectively integrate global-level information to alleviate issues associated with large motion. Specifically, we first estimate a pair of initial intermediate flows using a high-resolution feature map for extracting local details. Then, we incorporate a sparse global matching branch to compensate for flow estimation, which consists of identifying flaws in initial flows and generating sparse flow compensation with a global receptive field. Finally, we adaptively merge the initial flow estimation with global flow compensation, yielding a more accurate intermediate flow. To evaluate the effectiveness of our method in handling large motion, we carefully curate a more challenging subset from commonly used benchmarks. Our method demonstrates the state-of-the-art performance on these VFI subsets with large motion.