# SOSNet: Second Order Similarity Regularization for Local Descriptor Learning

arXiv: https://arxiv.org/abs/1904.05019 · HTML: https://mlsanity.com/paper/1904.05019/

## Key facts

- Cited by 131 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2019.

## Built on by (most cited first, 30 of 131)

- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [COTR: Correspondence Transformer for Matching Across Images](https://mlsanity.com/paper/2103.14167/) (2021)
- [ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation](https://mlsanity.com/paper/2304.03608/) (2023)
- [Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed](https://mlsanity.com/paper/2403.04765/) (2024)
- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction](https://mlsanity.com/paper/2112.02906/) (2021)
- [CROSSFIRE: Camera Relocalization On Self-Supervised Features from an Implicit Representation](https://mlsanity.com/paper/2303.04869/) (2023)
- [P2-Net: Joint Description and Detection of Local Features for Pixel and Point Matching](https://mlsanity.com/paper/2103.01055/) (2021)
- [MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training](https://mlsanity.com/paper/2501.07556/) (2025)
- [SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding](https://mlsanity.com/paper/2306.05407/) (2023)
- [TransforMatcher: Match-to-Match Attention for Semantic Correspondence](https://mlsanity.com/paper/2205.11634/) (2022)
- [Motion Basis Learning for Unsupervised Deep Homography Estimation with Subspace Projection](https://mlsanity.com/paper/2103.15346/) (2021)
- [Depth-Aware Multi-Grid Deep Homography Estimation with Contextual Correlation](https://mlsanity.com/paper/2107.02524/) (2021)
- [Decoupling Makes Weakly Supervised Local Feature Better](https://mlsanity.com/paper/2201.02861/) (2022)
- [SFD2: Semantic-guided Feature Detection and Description](https://mlsanity.com/paper/2304.14845/) (2023)
- [Semi-Supervised Keypoint Detector and Descriptor for Retinal Image Matching](https://mlsanity.com/paper/2207.07932/) (2022)
- [Self-Supervised Equivariant Learning for Oriented Keypoint Detection](https://mlsanity.com/paper/2204.08613/) (2022)
- [ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning](https://mlsanity.com/paper/2205.07439/) (2022)
- [Local Feature Matching Using Deep Learning: A Survey](https://mlsanity.com/paper/2401.17592/) (2024)
- [Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences](https://mlsanity.com/paper/2404.06337/) (2024)
- [Learning Rotation-Equivariant Features for Visual Correspondence](https://mlsanity.com/paper/2303.15472/) (2023)
- [NinjaDesc: Content-Concealing Visual Descriptors via Adversarial Learning](https://mlsanity.com/paper/2112.12785/) (2021)
- [Steerers: A framework for rotation equivariant keypoint descriptors](https://mlsanity.com/paper/2312.02152/) (2023)
- [Unsupervised Homography Estimation with Coplanarity-Aware GAN](https://mlsanity.com/paper/2205.03821/) (2022)
- [ECO-TR: Efficient Correspondences Finding Via Coarse-to-Fine Refinement](https://mlsanity.com/paper/2209.12213/) (2022)
- [Neural Reprojection Error: Merging Feature Learning and Camera Pose Estimation](https://mlsanity.com/paper/2103.07153/) (2021)
- [DFM: A Performance Baseline for Deep Feature Matching](https://mlsanity.com/paper/2106.07791/) (2021)
- [Improving Transformer-based Image Matching by Cascaded Capturing Spatially Informative Keypoints](https://mlsanity.com/paper/2303.02885/) (2023)
- [Segment Anything Model is a Good Teacher for Local Feature Learning](https://mlsanity.com/paper/2309.16992/) (2023)

## Abstract (arXiv)

Despite the fact that Second Order Similarity (SOS) has been used with significant success in tasks such as graph matching and clustering, it has not been exploited for learning local descriptors. In this work, we explore the potential of SOS in the field of descriptor learning by building upon the intuition that a positive pair of matching points should exhibit similar distances with respect to other points in the embedding space. Thus, we propose a novel regularization term, named Second Order Similarity Regularization (SOSR), that follows this principle. By incorporating SOSR into training, our learned descriptor achieves state-of-the-art performance on several challenging benchmarks containing distinct tasks ranging from local patch retrieval to structure from motion. Furthermore, by designing a von Mises-Fischer distribution based evaluation method, we link the utilization of the descriptor space to the matching performance, thus demonstrating the effectiveness of our proposed SOSR. Extensive experimental results, empirical evidence, and in-depth analysis are provided, indicating that SOSR can significantly boost the matching performance of the learned descriptor.