# Local Descriptors Optimized for Average Precision

arXiv: https://arxiv.org/abs/1804.05312 · HTML: https://mlsanity.com/paper/1804.05312/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2018.

## Built on by (most cited first, 30 of 43)

- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction](https://mlsanity.com/paper/2112.02906/) (2021)
- [DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features](https://mlsanity.com/paper/2108.02927/) (2021)
- [Recall@k Surrogate Loss with Large Batches and Similarity Mixup](https://mlsanity.com/paper/2108.11179/) (2021)
- [Large-Scale Spatio-Temporal Person Re-identification: Algorithms and Benchmark](https://mlsanity.com/paper/2105.15076/) (2021)
- [Global Features are All You Need for Image Retrieval and Reranking](https://mlsanity.com/paper/2308.06954/) (2023)
- [Learning Intra-Batch Connections for Deep Metric Learning](https://mlsanity.com/paper/2102.07753/) (2021)
- [SFD2: Semantic-guided Feature Detection and Description](https://mlsanity.com/paper/2304.14845/) (2023)
- [Image Clustering Conditioned on Text Criteria](https://mlsanity.com/paper/2310.18297/) (2023)
- [Local Feature Matching Using Deep Learning: A Survey](https://mlsanity.com/paper/2401.17592/) (2024)
- [Robust and Decomposable Average Precision for Image Retrieval](https://mlsanity.com/paper/2110.01445/) (2021)
- [AstroVision: Towards Autonomous Feature Detection and Description for Missions to Small Bodies Using Deep Learning](https://mlsanity.com/paper/2208.02053/) (2022)
- [FeatureBooster: Boosting Feature Descriptors with a Lightweight Neural Network](https://mlsanity.com/paper/2211.15069/) (2022)
- [LDP-Feat: Image Features with Local Differential Privacy](https://mlsanity.com/paper/2308.11223/) (2023)
- [Optimization of Rank Losses for Image Retrieval](https://mlsanity.com/paper/2309.08250/) (2023)
- [MetricOpt: Learning to Optimize Black-Box Evaluation Metrics](https://mlsanity.com/paper/2104.10631/) (2021)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [DALG: Deep Attentive Local and Global Modeling for Image Retrieval](https://mlsanity.com/paper/2207.00287/) (2022)
- [SDGMNet: Statistic-based Dynamic Gradient Modulation for Local Descriptor Learning](https://mlsanity.com/paper/2106.04434/) (2021)
- [Object-Centric Open-Vocabulary Image-Retrieval with Aggregated Features](https://mlsanity.com/paper/2309.14999/) (2023)
- [Rethinking the Optimization of Average Precision: Only Penalizing Negative Instances before Positive Ones is Enough](https://mlsanity.com/paper/2102.04640/) (2021)
- [Coarse-to-Fine: Learning Compact Discriminative Representation for Single-Stage Image Retrieval](https://mlsanity.com/paper/2308.04008/) (2023)
- [A Detector-oblivious Multi-arm Network for Keypoint Matching](https://mlsanity.com/paper/2104.00947/) (2021)
- [Revisiting Binary Local Image Description for Resource Limited Devices](https://mlsanity.com/paper/2108.08380/) (2021)
- [Digging Into Self-Supervised Learning of Feature Descriptors](https://mlsanity.com/paper/2110.04773/) (2021)
- [BEBLID: Boosted efficient binary local image descriptor](https://mlsanity.com/paper/2402.04482/) (2024)
- [ReF -- Rotation Equivariant Features for Local Feature Matching](https://mlsanity.com/paper/2203.05206/) (2022)
- [Descriptor Distillation: a Teacher-Student-Regularized Framework for Learning Local Descriptors](https://mlsanity.com/paper/2209.11795/) (2022)
- [Registering Neural Radiance Fields as 3D Density Images](https://mlsanity.com/paper/2305.12843/) (2023)
- [Modality-Aware Feature Matching in Visual and Vision-Language Applications: A Comprehensive Survey](https://mlsanity.com/paper/2507.22791/) (2025)

## Abstract (arXiv)

Extraction of local feature descriptors is a vital stage in the solution pipelines for numerous computer vision tasks. Learning-based approaches improve performance in certain tasks, but still cannot replace handcrafted features in general. In this paper, we improve the learning of local feature descriptors by optimizing the performance of descriptor matching, which is a common stage that follows descriptor extraction in local feature based pipelines, and can be formulated as nearest neighbor retrieval. Specifically, we directly optimize a ranking-based retrieval performance metric, Average Precision, using deep neural networks. This general-purpose solution can also be viewed as a listwise learning to rank approach, which is advantageous compared to recent local ranking approaches. On standard benchmarks, descriptors learned with our formulation achieve state-of-the-art results in patch verification, patch retrieval, and image matching.