# OmniGlue: Generalizable Feature Matching with Foundation Model Guidance

arXiv: https://arxiv.org/abs/2405.12979 · HTML: https://mlsanity.com/paper/2405.12979/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 59 references that are in the map.
- First posted on arXiv 21 May 2024.

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields](https://mlsanity.com/paper/2304.06706/) (2023)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age](https://mlsanity.com/paper/1606.05830/) (2016)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking](https://mlsanity.com/paper/1803.11285/) (2018)
- [DKM: Dense Kernelized Feature Matching for Geometry Estimation](https://mlsanity.com/paper/2202.00667/) (2022)
- [ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer](https://mlsanity.com/paper/2208.14201/) (2022)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [MatchFormer: Interleaving Attention in Transformers for Feature Matching](https://mlsanity.com/paper/2203.09645/) (2022)
- [Unsupervised Semantic Correspondence Using Stable Diffusion](https://mlsanity.com/paper/2305.15581/) (2023)
- [Learning Accurate Dense Correspondences and When to Trust Them](https://mlsanity.com/paper/2101.01710/) (2021)
- [Learning to Match Features with Seeded Graph Matching Network](https://mlsanity.com/paper/2108.08771/) (2021)

## Built on by (most cited first, 30 of 56)

- [CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents](https://mlsanity.com/paper/2407.01511/) (2024)
- [Retrieval-augmented Few-shot Medical Image Segmentation with Foundation Models](https://mlsanity.com/paper/2408.08813/) (2024)
- [MATCHA:Towards Matching Anything](https://mlsanity.com/paper/2501.14945/) (2025)
- [LiftFeat: 3D Geometry-Aware Local Feature Matching](https://mlsanity.com/paper/2505.03422/) (2025)
- [JOG3R: Towards 3D-Consistent Video Generators](https://mlsanity.com/paper/2501.01409/) (2025)
- [One-Shot Dual-Arm Imitation Learning](https://mlsanity.com/paper/2503.06831/) (2025)
- [Can Generative Video Models Help Pose Estimation?](https://mlsanity.com/paper/2412.16155/) (2024)
- [EDM: Efficient Deep Feature Matching](https://mlsanity.com/paper/2503.05122/) (2025)
- [CoMatcher: Multi-View Collaborative Feature Matching](https://mlsanity.com/paper/2504.01872/) (2025)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [A Simple yet Effective Test-Time Adaptation for Zero-Shot Monocular Metric Depth Estimation](https://mlsanity.com/paper/2412.14103/) (2024)
- [Leveraging Semantic Cues from Foundation Vision Models for Enhanced Local Feature Correspondence](https://mlsanity.com/paper/2410.09533/) (2024)
- [MapGlue: Multimodal Remote Sensing Image Matching](https://mlsanity.com/paper/2503.16185/) (2025)
- [DINO-VO: A Feature-based Visual Odometry Leveraging a Visual Foundation Model](https://mlsanity.com/paper/2507.13145/) (2025)
- [Mismatched: Evaluating the Limits of Image Matching Approaches and Benchmarks](https://mlsanity.com/paper/2408.16445/) (2024)
- [OpenMonoGS-SLAM: Monocular Gaussian Splatting SLAM with Open-set Semantics](https://mlsanity.com/paper/2512.08625/) (2025)
- [Long-tail Internet photo reconstruction](https://mlsanity.com/paper/2604.22714/) (2026)
- [A2-GNN: Angle-Annular GNN for Visual Descriptor-free Camera Relocalization](https://mlsanity.com/paper/2502.20036/) (2025)
- [CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registration](https://mlsanity.com/paper/2604.05689/) (2026)
- [Scalable and Generalizable Correspondence Pruning via Geometry-Consistent Pre-training](https://mlsanity.com/paper/2406.05773/) (2024)
- [AlignDiff: Learning Physically-Grounded Camera Alignment via Diffusion](https://mlsanity.com/paper/2503.21581/) (2025)
- [LightGlueStick: a Fast and Robust Glue for Joint Point-Line Matching](https://mlsanity.com/paper/2510.16438/) (2025)
- [SGMatch: Semantic-Guided Non-Rigid Shape Matching with Flow Regularization](https://mlsanity.com/paper/2603.12937/) (2026)
- [Are Pretrained Image Matchers Good Enough for SAR-Optical Satellite Registration?](https://mlsanity.com/paper/2604.10217/) (2026)
- [GIMS: Image Matching System Based on Adaptive Graph Construction and Graph Neural Network](https://mlsanity.com/paper/2412.18221/) (2024)
- [Explainable Coarse-to-Fine Ancient Manuscript Duplicates Discovery](https://mlsanity.com/paper/2505.03836/) (2025)
- [ZeroReg3D: A Zero-shot Registration Pipeline for 3D Consecutive Histopathology Image Reconstruction](https://mlsanity.com/paper/2506.21923/) (2025)
- [Multi-Order Matching Network for Alignment-Free Depth Super-Resolution](https://mlsanity.com/paper/2511.16361/) (2025)
- [Understanding and Optimizing Attention-Based Sparse Matching for Diverse Local Features](https://mlsanity.com/paper/2602.08430/) (2026)
- [FlowFixer: Towards Detail-Preserving Subject-Driven Generation](https://mlsanity.com/paper/2602.21402/) (2026)

## Abstract (arXiv)

The image matching field has been witnessing a continuous emergence of novel learnable feature matching techniques, with ever-improving performance on conventional benchmarks. However, our investigation shows that despite these gains, their potential for real-world applications is restricted by their limited generalization capabilities to novel image domains. In this paper, we introduce OmniGlue, the first learnable image matcher that is designed with generalization as a core principle. OmniGlue leverages broad knowledge from a vision foundation model to guide the feature matching process, boosting generalization to domains not seen at training time. Additionally, we propose a novel keypoint position-guided attention mechanism which disentangles spatial and appearance information, leading to enhanced matching descriptors. We perform comprehensive experiments on a suite of 7 datasets with varied image domains, including scene-level, object-centric and aerial images. OmniGlue's novel components lead to relative gains on unseen domains of 20.9% with respect to a directly comparable reference model, while also outperforming the recent LightGlue method by 9.5% relatively.Code and model can be found at https://hwjiang1510.github.io/OmniGlue