# Video Frame Interpolation with Densely Queried Bilateral Correlation

arXiv: https://arxiv.org/abs/2304.13596 · HTML: https://mlsanity.com/paper/2304.13596/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 51 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 30 of 33)

- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [View Synthesis by Appearance Flow](https://mlsanity.com/paper/1605.03557/) (2016)
- [Video Frame Interpolation via Adaptive Convolution](https://mlsanity.com/paper/1703.07514/) (2017)
- [Context-aware Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/1803.10967/) (2018)
- [FILM: Frame Interpolation for Large Motion](https://mlsanity.com/paper/2202.04901/) (2022)
- [TimeLens: Event-based Video Frame Interpolation](https://mlsanity.com/paper/2106.07286/) (2021)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Quadratic video interpolation](https://mlsanity.com/paper/1911.00627/) (2019)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation](https://mlsanity.com/paper/1907.10244/) (2019)
- [XVFI: eXtreme Video Frame Interpolation](https://mlsanity.com/paper/2103.16206/) (2021)
- [Video Frame Interpolation Transformer](https://mlsanity.com/paper/2111.13817/) (2021)
- [BMBC:Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation](https://mlsanity.com/paper/2007.12622/) (2020)
- [IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation](https://mlsanity.com/paper/2205.14620/) (2022)
- [Asymmetric Bilateral Motion Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2108.06815/) (2021)
- [Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution](https://mlsanity.com/paper/2002.11616/) (2020)
- [Deep Animation Video Interpolation in the Wild](https://mlsanity.com/paper/2104.02495/) (2021)
- [Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolution](https://mlsanity.com/paper/2006.08070/) (2020)
- [PhaseNet for Video Frame Interpolation](https://mlsanity.com/paper/1804.00884/) (2018)
- [Space-Time-Aware Multi-Resolution Video Enhancement](https://mlsanity.com/paper/2003.13170/) (2020)
- [Enhanced Quadratic Video Interpolation](https://mlsanity.com/paper/2009.04642/) (2020)
- [All at Once: Temporally Adaptive Multi-Frame Interpolation with Advanced Motion Modeling](https://mlsanity.com/paper/2007.11762/) (2020)
- [ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolation](https://mlsanity.com/paper/2111.15483/) (2021)
- [FISR: Deep Joint Frame Interpolation and Super-Resolution with a Multi-scale Temporal Loss](https://mlsanity.com/paper/1912.07213/) (2019)
- [Optimizing Video Prediction via Video Frame Interpolation](https://mlsanity.com/paper/2206.13454/) (2022)

## Built on by (most cited first, 8 of 8)

- [MoSt-DSA: Modeling Motion and Structural Interactions for Direct Multi-Frame Interpolation in DSA Images](https://mlsanity.com/paper/2407.07078/) (2024)
- [Video Face Re-Aging: Toward Temporally Consistent Face Re-Aging](https://mlsanity.com/paper/2311.11642/) (2023)
- [GaraMoSt: Parallel Multi-Granularity Motion and Structural Modeling for Efficient Multi-Frame Interpolation in DSA Images](https://mlsanity.com/paper/2412.14118/) (2024)
- [360VFI: A Dataset and Benchmark for Omnidirectional Video Frame Interpolation](https://mlsanity.com/paper/2407.14066/) (2024)
- [Bridging the Gap: Sketch-Aware Interpolation Network for High-Quality Animation Sketch Inbetweening](https://mlsanity.com/paper/2308.13273/) (2023)
- [HSTR-Net: Reference Based Video Super-resolution with Dual Cameras](https://mlsanity.com/paper/2310.12092/) (2023)
- [Restoration-Oriented Video Frame Interpolation with Region-Distinguishable Priors from SAM](https://mlsanity.com/paper/2312.15868/) (2023)
- [SVI360: Spherical Video Interpolation](https://mlsanity.com/paper/2607.11710/) (2026)

## Abstract (arXiv)

Video Frame Interpolation (VFI) aims to synthesize non-existent intermediate frames between existent frames. Flow-based VFI algorithms estimate intermediate motion fields to warp the existent frames. Real-world motions' complexity and the reference frame's absence make motion estimation challenging. Many state-of-the-art approaches explicitly model the correlations between two neighboring frames for more accurate motion estimation. In common approaches, the receptive field of correlation modeling at higher resolution depends on the motion fields estimated beforehand. Such receptive field dependency makes common motion estimation approaches poor at coping with small and fast-moving objects. To better model correlations and to produce more accurate motion fields, we propose the Densely Queried Bilateral Correlation (DQBC) that gets rid of the receptive field dependency problem and thus is more friendly to small and fast-moving objects. The motion fields generated with the help of DQBC are further refined and up-sampled with context features. After the motion fields are fixed, a CNN-based SynthNet synthesizes the final interpolated frame. Experiments show that our approach enjoys higher accuracy and less inference time than the state-of-the-art. Source code is available at https://github.com/kinoud/DQBC.