# Improving Calibration in Deep Metric Learning With Cross-Example Softmax

arXiv: https://arxiv.org/abs/2011.08824 · HTML: https://mlsanity.com/paper/2011.08824/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2020.

## Built on by (most cited first, 3 of 3)

- [ELM: Embedding and Logit Margins for Long-Tail Learning](https://mlsanity.com/paper/2204.13208/) (2022)
- [Threshold-Consistent Margin Loss for Open-World Deep Metric Learning](https://mlsanity.com/paper/2307.04047/) (2023)
- [Fine-grained auxiliary learning for real-world product recommendation](https://mlsanity.com/paper/2510.04551/) (2025)

## Abstract (arXiv)

Modern image retrieval systems increasingly rely on the use of deep neural networks to learn embedding spaces in which distance encodes the relevance between a given query and image. In this setting, existing approaches tend to emphasize one of two properties. Triplet-based methods capture top-k relevancy, where all top-k scoring documents are assumed to be relevant to a given query Pairwise contrastive models capture threshold relevancy, where all documents scoring higher than some threshold are assumed to be relevant. In this paper, we propose Cross-Example Softmax which combines the properties of top-k and threshold relevancy. In each iteration, the proposed loss encourages all queries to be closer to their matching images than all queries are to all non-matching images. This leads to a globally more calibrated similarity metric and makes distance more interpretable as an absolute measure of relevance. We further introduce Cross-Example Negative Mining, in which each pair is compared to the hardest negative comparisons across the entire batch. Empirically, we show in a series of experiments on Conceptual Captions and Flickr30k, that the proposed method effectively improves global calibration and also retrieval performance.