# Leveraging Semantic Cues from Foundation Vision Models for Enhanced Local Feature Correspondence

arXiv: https://arxiv.org/abs/2410.09533 · HTML: https://mlsanity.com/paper/2410.09533/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 25 references that are in the map.
- First posted on arXiv 12 Oct 2024.

## Builds on (most cited first, 22 of 22)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation](https://mlsanity.com/paper/2304.03608/) (2023)
- [Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed](https://mlsanity.com/paper/2403.04765/) (2024)
- [XFeat: Accelerated Features for Lightweight Image Matching](https://mlsanity.com/paper/2404.19174/) (2024)
- [ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction](https://mlsanity.com/paper/2112.02906/) (2021)
- [DeDoDe: Detect, Don't Describe -- Describe, Don't Detect for Local Feature Matching](https://mlsanity.com/paper/2308.08479/) (2023)
- [OmniGlue: Generalizable Feature Matching with Foundation Model Guidance](https://mlsanity.com/paper/2405.12979/) (2024)
- [SFD2: Semantic-guided Feature Detection and Description](https://mlsanity.com/paper/2304.14845/) (2023)
- [Learning Rotation-Equivariant Features for Visual Correspondence](https://mlsanity.com/paper/2303.15472/) (2023)
- [MESA: Matching Everything by Segmenting Anything](https://mlsanity.com/paper/2401.16741/) (2024)
- [Enhancing Deformable Local Features by Jointly Learning to Detect and Describe Keypoints](https://mlsanity.com/paper/2304.00583/) (2023)
- [Sim2Real Object-Centric Keypoint Detection and Description](https://mlsanity.com/paper/2202.00448/) (2022)
- [Extracting Deformation-Aware Local Features by Learning to Deform](https://mlsanity.com/paper/2111.10617/) (2021)
- [Learning Geodesic-Aware Local Features from RGB-D Images](https://mlsanity.com/paper/2203.12016/) (2022)

## Built on by (most cited first, 5 of 5)

- [Hierarchical Image Matching for UAV Absolute Visual Localization via Semantic and Structural Constraints](https://mlsanity.com/paper/2506.09748/) (2025)
- [Region Matters: Efficient and Reliable Region-Aware Visual Place Recognition](https://mlsanity.com/paper/2604.22390/) (2026)
- [AnyMatch: Supercharging Universal Multi-Modal Image Matching with Large-Scale Single-View Images](https://mlsanity.com/paper/2606.31077/) (2026)
- [Cross-View Feature Matching: Survey, Benchmarking, and Foundation-Model Perspectives](https://mlsanity.com/paper/2608.11093/) (2026)
- [Misanthrope: A Privacy-Preserving Keypoint Detector](https://mlsanity.com/paper/2608.23012/) (2026)

## Abstract (arXiv)

Visual correspondence is a crucial step in key computer vision tasks, including camera localization, image registration, and structure from motion. The most effective techniques for matching keypoints currently involve using learned sparse or dense matchers, which need pairs of images. These neural networks have a good general understanding of features from both images, but they often struggle to match points from different semantic areas. This paper presents a new method that uses semantic cues from foundation vision model features (like DINOv2) to enhance local feature matching by incorporating semantic reasoning into existing descriptors. Therefore, the learned descriptors do not require image pairs at inference time, allowing feature caching and fast matching using similarity search, unlike learned matchers. We present adapted versions of six existing descriptors, with an average increase in performance of 29% in camera localization, with comparable accuracy to existing matchers as LightGlue and LoFTR in two existing benchmarks. Both code and trained models are available at https://www.verlab.dcc.ufmg.br/descriptors/reasoning_accv24