# GIM: Learning Generalizable Image Matcher From Internet Videos

arXiv: https://arxiv.org/abs/2402.11095 · HTML: https://mlsanity.com/paper/2402.11095/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image](https://mlsanity.com/paper/2307.10984/) (2023)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [DKM: Dense Kernelized Feature Matching for Geometry Estimation](https://mlsanity.com/paper/2202.00667/) (2022)
- [MAGSAC: marginalizing sample consensus](https://mlsanity.com/paper/1803.07469/) (2018)
- [Flow-Motion and Depth Network for Monocular Stereo and Beyond](https://mlsanity.com/paper/1909.05452/) (2019)
- [Self-supervised Geometric Perception](https://mlsanity.com/paper/2103.03114/) (2021)
- [Matchable Image Retrieval by Learning from Surface Reconstruction](https://mlsanity.com/paper/1811.10343/) (2018)

## Built on by (most cited first, 30 of 75)

- [ReCamMaster: Camera-Controlled Generative Rendering from A Single Video](https://mlsanity.com/paper/2503.11647/) (2025)
- [SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpoints](https://mlsanity.com/paper/2412.07760/) (2024)
- [RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting Simulator](https://mlsanity.com/paper/2411.11839/) (2024)
- [CamCloneMaster: Enabling Reference-based Camera Control for Video Generation](https://mlsanity.com/paper/2506.03140/) (2025)
- [MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training](https://mlsanity.com/paper/2501.07556/) (2025)
- [UFM: A Simple Path towards Unified Dense Correspondence with Flow](https://mlsanity.com/paper/2506.09278/) (2025)
- [4Real-Video: Learning Generalizable Photo-Realistic 4D Video Diffusion](https://mlsanity.com/paper/2412.04462/) (2024)
- [Structure Consistent Gaussian Splatting with Matching Prior for Few-shot Novel View Synthesis](https://mlsanity.com/paper/2411.03637/) (2024)
- [LuSh-NeRF: Lighting up and Sharpening NeRFs for Low-light Scenes](https://mlsanity.com/paper/2411.06757/) (2024)
- [ReRoPE: Repurposing RoPE for Relative Camera Control](https://mlsanity.com/paper/2602.08068/) (2026)
- [Dynamic View Synthesis as an Inverse Problem](https://mlsanity.com/paper/2506.08004/) (2025)
- [OrthoLoC: UAV 6-DoF Localization and Calibration Using Orthographic Geodata](https://mlsanity.com/paper/2509.18350/) (2025)
- [EMMA: Generalizing Real-World Robot Manipulation via Generative Visual Transfer](https://mlsanity.com/paper/2509.22407/) (2025)
- [GV-Bench: Benchmarking Local Feature Matching for Geometric Verification of Long-term Loop Closure Detection](https://mlsanity.com/paper/2407.11736/) (2024)
- [VideoLifter: Lifting Videos to 3D with Fast Hierarchical Stereo Alignment](https://mlsanity.com/paper/2501.01949/) (2025)
- [CorrFill: Enhancing Faithfulness in Reference-based Inpainting with Correspondence Guidance in Diffusion Models](https://mlsanity.com/paper/2501.02355/) (2025)
- [One-Shot Dual-Arm Imitation Learning](https://mlsanity.com/paper/2503.06831/) (2025)
- [RoboTransfer: Controllable Geometry-Consistent Video Diffusion for Manipulation Policy Transfer](https://mlsanity.com/paper/2505.23171/) (2025)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [RoboVIP: Multi-View Video Generation with Visual Identity Prompting Augments Robot Manipulation](https://mlsanity.com/paper/2601.05241/) (2026)
- [CineScene: Implicit 3D as Effective Scene Representation for Cinematic Video Generation](https://mlsanity.com/paper/2602.06959/) (2026)
- [MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matching](https://mlsanity.com/paper/2501.11299/) (2025)
- [Martian World Model: Controllable Video Synthesis with Physically Accurate 3D Reconstructions](https://mlsanity.com/paper/2507.07978/) (2025)
- [MapGlue: Multimodal Remote Sensing Image Matching](https://mlsanity.com/paper/2503.16185/) (2025)
- [Learning Affine Correspondences by Integrating Geometric Constraints](https://mlsanity.com/paper/2504.04834/) (2025)
- [SegMASt3R: Geometry Grounded Segment Matching](https://mlsanity.com/paper/2510.05051/) (2025)
- [Full-4D: Generating Full-Scope 4D Scenes from a Single-View Video](https://mlsanity.com/paper/2605.25500/) (2026)
- [Mismatched: Evaluating the Limits of Image Matching Approaches and Benchmarks](https://mlsanity.com/paper/2408.16445/) (2024)
- [LiteVLoc: Map-Lite Visual Localization for Image Goal Navigation](https://mlsanity.com/paper/2410.04419/) (2024)
- [Learning Dense Feature Matching via Lifting Single 2D Image to 3D Space](https://mlsanity.com/paper/2507.00392/) (2025)

## Abstract (arXiv)

Image matching is a fundamental computer vision problem. While learning-based methods achieve state-of-the-art performance on existing benchmarks, they generalize poorly to in-the-wild images. Such methods typically need to train separate models for different scene types and are impractical when the scene type is unknown in advance. One of the underlying problems is the limited scalability of existing data construction pipelines, which limits the diversity of standard image matching datasets. To address this problem, we propose GIM, a self-training framework for learning a single generalizable model based on any image matching architecture using internet videos, an abundant and diverse data source. Given an architecture, GIM first trains it on standard domain-specific datasets and then combines it with complementary matching methods to create dense labels on nearby frames of novel videos. These labels are filtered by robust fitting, and then enhanced by propagating them to distant frames. The final model is trained on propagated data with strong augmentations. We also propose ZEB, the first zero-shot evaluation benchmark for image matching. By mixing data from diverse domains, ZEB can thoroughly assess the cross-domain generalization performance of different methods. Applying GIM consistently improves the zero-shot performance of 3 state-of-the-art image matching architectures; with 50 hours of YouTube videos, the relative zero-shot performance improves by 8.4%-18.1%. GIM also enables generalization to extreme cross-domain data such as Bird Eye View (BEV) images of projected 3D point clouds (Fig. 1(c)). More importantly, our single zero-shot model consistently outperforms domain-specific baselines when evaluated on downstream tasks inherent to their respective domains. The video presentation is available at https://www.youtube.com/watch?v=FU_MJLD8LeY.