# Learning with Average Precision: Training Image Retrieval with a Listwise Loss

arXiv: https://arxiv.org/abs/1906.07589 · HTML: https://mlsanity.com/paper/1906.07589/

## Key facts

- Cited by 185 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2019.

## Built on by (most cited first, 30 of 185)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motion](https://mlsanity.com/paper/2409.19152/) (2024)
- [CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion](https://mlsanity.com/paper/2210.10716/) (2022)
- [Accelerated Coordinate Encoding: Learning to Relocalize in Minutes using RGB and Poses](https://mlsanity.com/paper/2305.14059/) (2023)
- [TransVPR: Transformer-based place recognition with multi-level attention aggregation](https://mlsanity.com/paper/2201.02001/) (2022)
- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [Training Vision Transformers for Image Retrieval](https://mlsanity.com/paper/2102.05644/) (2021)
- [Deep Visual Geo-localization Benchmark](https://mlsanity.com/paper/2204.03444/) (2022)
- [Where is your place, Visual Place Recognition?](https://mlsanity.com/paper/2103.06443/) (2021)
- [Deep Learning for Instance Retrieval: A Survey](https://mlsanity.com/paper/2101.11282/) (2021)
- [Instance-level Image Retrieval using Reranking Transformers](https://mlsanity.com/paper/2103.12236/) (2021)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [MegaLoc: One Retrieval to Place Them All](https://mlsanity.com/paper/2502.17237/) (2025)
- [SeqNet: Learning Descriptors for Sequence-based Hierarchical Place Recognition](https://mlsanity.com/paper/2102.11603/) (2021)
- [DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features](https://mlsanity.com/paper/2108.02927/) (2021)
- [RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression](https://mlsanity.com/paper/2205.15236/) (2022)
- [Recall@k Surrogate Loss with Large Batches and Similarity Mixup](https://mlsanity.com/paper/2108.11179/) (2021)
- [Large-Scale Spatio-Temporal Person Re-identification: Algorithms and Benchmark](https://mlsanity.com/paper/2105.15076/) (2021)
- [Zero-Shot Day-Night Domain Adaptation with a Physics Prior](https://mlsanity.com/paper/2108.05137/) (2021)
- [Correlation Verification for Image Retrieval](https://mlsanity.com/paper/2204.01458/) (2022)
- [GeneCIS: A Benchmark for General Conditional Image Similarity](https://mlsanity.com/paper/2306.07969/) (2023)
- [On Semantic Similarity in Video Retrieval](https://mlsanity.com/paper/2103.10095/) (2021)
- [Improving Point Cloud Based Place Recognition with Ranking-based Loss and Large Batch Training](https://mlsanity.com/paper/2203.00972/) (2022)
- [Data-efficient Large Scale Place Recognition with Graded Similarity Supervision](https://mlsanity.com/paper/2303.11739/) (2023)
- [MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition](https://mlsanity.com/paper/2104.05327/) (2021)
- [Global Features are All You Need for Image Retrieval and Reranking](https://mlsanity.com/paper/2308.06954/) (2023)
- [Single Image Depth Estimation: An Overview](https://mlsanity.com/paper/2104.06456/) (2021)

## Abstract (arXiv)

Image retrieval can be formulated as a ranking problem where the goal is to order database images by decreasing similarity to the query. Recent deep models for image retrieval have outperformed traditional methods by leveraging ranking-tailored loss functions, but important theoretical and practical problems remain. First, rather than directly optimizing the global ranking, they minimize an upper-bound on the essential loss, which does not necessarily result in an optimal mean average precision (mAP). Second, these methods require significant engineering efforts to work well, e.g. special pre-training and hard-negative mining. In this paper we propose instead to directly optimize the global mAP by leveraging recent advances in listwise loss formulations. Using a histogram binning approximation, the AP can be differentiated and thus employed to end-to-end learning. Compared to existing losses, the proposed method considers thousands of images simultaneously at each iteration and eliminates the need for ad hoc tricks. It also establishes a new state of the art on many standard retrieval benchmarks. Models and evaluation scripts have been made available at https://europe.naverlabs.com/Deep-Image-Retrieval/