# Learning Image Matching by Simply Watching Video

arXiv: https://arxiv.org/abs/1603.06041 · HTML: https://mlsanity.com/paper/1603.06041/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2016.

## Built on by (most cited first, 30 of 41)

- [Asymmetric Bilateral Motion Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2108.06815/) (2021)
- [Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation](https://mlsanity.com/paper/2303.00440/) (2023)
- [CDFI: Compression-Driven Network Design for Frame Interpolation](https://mlsanity.com/paper/2103.10559/) (2021)
- [ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolation](https://mlsanity.com/paper/2111.15483/) (2021)
- [BiFormer: Learning Bilateral Motion Estimation via Bilateral Transformer for 4K Video Frame Interpolation](https://mlsanity.com/paper/2304.02225/) (2023)
- [Video Interpolation by Event-driven Anisotropic Adjustment of Optical Flow](https://mlsanity.com/paper/2208.09127/) (2022)
- [NTIRE 2021 Challenge on Video Super-Resolution](https://mlsanity.com/paper/2104.14852/) (2021)
- [A Perceptual Quality Metric for Video Frame Interpolation](https://mlsanity.com/paper/2210.01879/) (2022)
- [Learning Cross-Video Neural Representations for High-Quality Frame Interpolation](https://mlsanity.com/paper/2203.00137/) (2022)
- [Cross-Attention Transformer for Video Interpolation](https://mlsanity.com/paper/2207.04132/) (2022)
- [Towards Interpretable Video Super-Resolution via Alternating Optimization](https://mlsanity.com/paper/2207.10765/) (2022)
- [Efficient Space-time Video Super Resolution using Low-Resolution Flow and Mask Upsampling](https://mlsanity.com/paper/2104.05778/) (2021)
- [FastRIFE: Optimization of Real-Time Intermediate Flow Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2105.13482/) (2021)
- [AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation](https://mlsanity.com/paper/2506.01061/) (2025)
- [Long-term Video Frame Interpolation via Feature Propagation](https://mlsanity.com/paper/2203.15427/) (2022)
- [Enhanced Correlation Matching based Video Frame Interpolation](https://mlsanity.com/paper/2111.08869/) (2021)
- [ATCA: an Arc Trajectory Based Model with Curvature Attention for Video Frame Interpolation](https://mlsanity.com/paper/2208.00856/) (2022)
- [Sparsity-guided Network Design for Frame Interpolation](https://mlsanity.com/paper/2209.04551/) (2022)
- [Doduo: Learning Dense Visual Correspondence from Unsupervised Semantic-Aware Flow](https://mlsanity.com/paper/2309.15110/) (2023)
- [Zooming SlowMo: An Efficient One-Stage Framework for Space-Time Video Super-Resolution](https://mlsanity.com/paper/2104.07473/) (2021)
- [Texture-aware Video Frame Interpolation](https://mlsanity.com/paper/2102.13520/) (2021)
- [Exemplar-based Video Colorization with Long-term Spatiotemporal Dependency](https://mlsanity.com/paper/2303.15081/) (2023)
- [Ada-VSR: Adaptive Video Super-Resolution with Meta-Learning](https://mlsanity.com/paper/2108.02832/) (2021)
- [SPColor: Semantic Prior Guided Exemplar-based Image Colorization](https://mlsanity.com/paper/2304.06255/) (2023)
- [Event-based Video Frame Interpolation with Edge Guided Motion Refinement](https://mlsanity.com/paper/2404.18156/) (2024)
- [Render In-between: Motion Guided Video Synthesis for Action Interpolation](https://mlsanity.com/paper/2111.01029/) (2021)
- [Learning Spatio-Temporal Downsampling for Effective Video Upscaling](https://mlsanity.com/paper/2203.08140/) (2022)
- [Beyond a Video Frame Interpolator: A Space Decoupled Learning Approach to Continuous Image Transition](https://mlsanity.com/paper/2203.09771/) (2022)
- [Efficient Feature Extraction for High-resolution Video Frame Interpolation](https://mlsanity.com/paper/2211.14005/) (2022)
- [Temporal Spatial-Adaptive Interpolation with Deformable Refinement for Electron Microscopic Images](https://mlsanity.com/paper/2101.06771/) (2021)

## Abstract (arXiv)

This work presents an unsupervised learning based approach to the ubiquitous computer vision problem of image matching. We start from the insight that the problem of frame-interpolation implicitly solves for inter-frame correspondences. This permits the application of analysis-by-synthesis: we firstly train and apply a Convolutional Neural Network for frame-interpolation, then obtain correspondences by inverting the learned CNN. The key benefit behind this strategy is that the CNN for frame-interpolation can be trained in an unsupervised manner by exploiting the temporal coherency that is naturally contained in real-world video sequences. The present model therefore learns image matching by simply watching videos. Besides a promise to be more generally applicable, the presented approach achieves surprising performance comparable to traditional empirically designed methods.