# Unbiased Evaluation of Deep Metric Learning Algorithms

arXiv: https://arxiv.org/abs/1911.12528 · HTML: https://mlsanity.com/paper/1911.12528/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2019.

## Built on by (most cited first, 10 of 10)

- [Learning with Memory-based Virtual Classes for Deep Metric Learning](https://mlsanity.com/paper/2103.16940/) (2021)
- [Integrating Language Guidance into Vision-based Deep Metric Learning](https://mlsanity.com/paper/2203.08543/) (2022)
- [Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning](https://mlsanity.com/paper/2103.15454/) (2021)
- [Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning](https://mlsanity.com/paper/2107.09562/) (2021)
- [Granularity-aware Adaptation for Image Retrieval over Multiple Tasks](https://mlsanity.com/paper/2210.02254/) (2022)
- [Deep Metric Learning with Chance Constraints](https://mlsanity.com/paper/2209.09060/) (2022)
- [Metric Learning for Session-based Recommendations](https://mlsanity.com/paper/2101.02655/) (2021)
- [Generalized Sum Pooling for Metric Learning](https://mlsanity.com/paper/2308.09228/) (2023)
- [All You Need to Know About Training Image Retrieval Models](https://mlsanity.com/paper/2503.13045/) (2025)
- [Simple and Effective Balance of Contrastive Losses](https://mlsanity.com/paper/2112.11743/) (2021)

## Abstract (arXiv)

Deep metric learning (DML) is a popular approach for images retrieval, solving verification (same or not) problems and addressing open set classification. Arguably, the most common DML approach is with triplet loss, despite significant advances in the area of DML. Triplet loss suffers from several issues such as collapse of the embeddings, high sensitivity to sampling schemes and more importantly a lack of performance when compared to more modern methods. We attribute this adoption to a lack of fair comparisons between various methods and the difficulty in adopting them for novel problem statements. In this paper, we perform an unbiased comparison of the most popular DML baseline methods under same conditions and more importantly, not obfuscating any hyper parameter tuning or adjustment needed to favor a particular method. We find, that under equal conditions several older methods perform significantly better than previously believed. In fact, our unified implementation of 12 recently introduced DML algorithms achieve state-of-the art performance on CUB200, CAR196, and Stanford Online products datasets which establishes a new set of baselines for future DML research. The codebase and all tuned hyperparameters will be open-sourced for reproducibility and to serve as a source of benchmark.