# Do Different Deep Metric Learning Losses Lead to Similar Learned Features?

arXiv: https://arxiv.org/abs/2205.02698 · HTML: https://mlsanity.com/paper/2205.02698/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 5 May 2022.

## Builds on (most cited first, 30 of 30)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning](https://mlsanity.com/paper/1904.06627/) (2019)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [Additive Margin Softmax for Face Verification](https://mlsanity.com/paper/1801.05599/) (2018)
- [NormFace: L2 Hypersphere Embedding for Face Verification](https://mlsanity.com/paper/1704.06369/) (2017)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [SoftTriple Loss: Deep Metric Learning Without Triplet Sampling](https://mlsanity.com/paper/1909.05235/) (2019)
- [Deep Metric Learning with Hierarchical Triplet Loss](https://mlsanity.com/paper/1810.06951/) (2018)
- [Interpreting Deep Visual Representations via Network Dissection](https://mlsanity.com/paper/1711.05611/) (2017)
- [Revisiting Training Strategies and Generalization Performance in Deep Metric Learning](https://mlsanity.com/paper/2002.08473/) (2020)
- [Attribution in Scale and Space](https://mlsanity.com/paper/2004.03383/) (2020)
- [MIC: Mining Interclass Characteristics for Improved Metric Learning](https://mlsanity.com/paper/1909.11574/) (2019)
- [Signal-to-Noise Ratio: A Robust Distance Metric for Deep Metric Learning](https://mlsanity.com/paper/1904.02616/) (2019)
- [Visualizing Deep Similarity Networks](https://mlsanity.com/paper/1901.00536/) (2019)
- [Adapting Grad-CAM for Embedding Networks](https://mlsanity.com/paper/2001.06538/) (2020)
- [Visual Explanation for Deep Metric Learning](https://mlsanity.com/paper/1909.12977/) (2019)
- [MapLUR: Exploring a new Paradigm for Estimating Air Pollution using Deep Learning on Map Images](https://mlsanity.com/paper/2002.07493/) (2020)

## Built on by (most cited first, 6 of 6)

- [InDiReCT: Language-Guided Zero-Shot Deep Metric Learning for Images](https://mlsanity.com/paper/2211.12760/) (2022)
- [Data-Side Efficiencies for Lightweight Convolutional Neural Networks](https://mlsanity.com/paper/2308.13057/) (2023)
- [On Background Bias in Deep Metric Learning](https://mlsanity.com/paper/2210.01615/) (2022)
- [Getting More for Less: Using Weak Labels and AV-Mixup for Robust Audio-Visual Speaker Verification](https://mlsanity.com/paper/2309.07115/) (2023)
- [Training-free Conditional Image Embedding Framework Leveraging Large Vision Language Models](https://mlsanity.com/paper/2512.21860/) (2025)
- [Transparency Techniques for Neural Networks trained on Writer Identification and Writer Verification](https://mlsanity.com/paper/2506.16331/) (2025)

## Abstract (arXiv)

Recent studies have shown that many deep metric learning loss functions perform very similarly under the same experimental conditions. One potential reason for this unexpected result is that all losses let the network focus on similar image regions or properties. In this paper, we investigate this by conducting a two-step analysis to extract and compare the learned visual features of the same model architecture trained with different loss functions: First, we compare the learned features on the pixel level by correlating saliency maps of the same input images. Second, we compare the clustering of embeddings for several image properties, e.g. object color or illumination. To provide independent control over these properties, photo-realistic 3D car renders similar to images in the Cars196 dataset are generated. In our analysis, we compare 14 pretrained models from a recent study and find that, even though all models perform similarly, different loss functions can guide the model to learn different features. We especially find differences between classification and ranking based losses. Our analysis also shows that some seemingly irrelevant properties can have significant influence on the resulting embedding. We encourage researchers from the deep metric learning community to use our methods to get insights into the features learned by their proposed methods.