# SoftTriple Loss: Deep Metric Learning Without Triplet Sampling

arXiv: https://arxiv.org/abs/1909.05235 · HTML: https://mlsanity.com/paper/1909.05235/

## Key facts

- Cited by 163 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2019.

## Built on by (most cited first, 30 of 163)

- [Hyperbolic Vision Transformers: Combining Improvements in Metric Learning](https://mlsanity.com/paper/2203.10833/) (2022)
- [Training Vision Transformers for Image Retrieval](https://mlsanity.com/paper/2102.05644/) (2021)
- [Are All Losses Created Equal: A Neural Collapse Perspective](https://mlsanity.com/paper/2210.02192/) (2022)
- [Recall@k Surrogate Loss with Large Batches and Similarity Mixup](https://mlsanity.com/paper/2108.11179/) (2021)
- [von Mises-Fisher Loss: An Exploration of Embedding Geometries for Supervised Learning](https://mlsanity.com/paper/2103.15718/) (2021)
- [Unicom: Universal and Compact Representation Learning for Image Retrieval](https://mlsanity.com/paper/2304.05884/) (2023)
- [Learning Intra-Batch Connections for Deep Metric Learning](https://mlsanity.com/paper/2102.07753/) (2021)
- [UFPMP-Det: Toward Accurate and Efficient Object Detection on Drone Imagery](https://mlsanity.com/paper/2112.10415/) (2021)
- [Non-isotropy Regularization for Proxy-based Deep Metric Learning](https://mlsanity.com/paper/2203.08547/) (2022)
- [VS-Net: Voting with Segmentation for Visual Localization](https://mlsanity.com/paper/2105.10886/) (2021)
- [Unsupervised Visual Representation Learning by Online Constrained K-Means](https://mlsanity.com/paper/2105.11527/) (2021)
- [Intra-Modal Proxy Learning for Zero-Shot Visual Categorization with CLIP](https://mlsanity.com/paper/2310.19752/) (2023)
- [HIER: Metric Learning Beyond Class Labels via Hierarchical Regularization](https://mlsanity.com/paper/2212.14258/) (2022)
- [Query-by-Example Keyword Spotting system using Multi-head Attention and Softtriple Loss](https://mlsanity.com/paper/2102.07061/) (2021)
- [Stable Cluster Discrimination for Deep Clustering](https://mlsanity.com/paper/2311.14310/) (2023)
- [Noise-resistant Deep Metric Learning with Ranking-based Instance Selection](https://mlsanity.com/paper/2103.16047/) (2021)
- [It Takes Two to Tango: Mixup for Deep Metric Learning](https://mlsanity.com/paper/2106.04990/) (2021)
- [SphereFace Revived: Unifying Hyperspherical Face Recognition](https://mlsanity.com/paper/2109.05565/) (2021)
- [Deep Relational Metric Learning](https://mlsanity.com/paper/2108.10026/) (2021)
- [Delving into Semantic Scale Imbalance](https://mlsanity.com/paper/2212.14613/) (2022)
- [Online Zero-Shot Classification with CLIP](https://mlsanity.com/paper/2408.13320/) (2024)
- [Embedding Transfer with Label Relaxation for Improved Metric Learning](https://mlsanity.com/paper/2103.14908/) (2021)
- [Learning with Memory-based Virtual Classes for Deep Metric Learning](https://mlsanity.com/paper/2103.16940/) (2021)
- [Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2104.01286/) (2021)
- [Multi-Modal Proxy Learning Towards Personalized Visual Multiple Clustering](https://mlsanity.com/paper/2404.15655/) (2024)
- [Integrating Language Guidance into Vision-based Deep Metric Learning](https://mlsanity.com/paper/2203.08543/) (2022)
- [A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learning](https://mlsanity.com/paper/2207.03784/) (2022)
- [Satellite Image Based Cross-view Localization for Autonomous Vehicle](https://mlsanity.com/paper/2207.13506/) (2022)
- [Hierarchical Proxy-based Loss for Deep Metric Learning](https://mlsanity.com/paper/2103.13538/) (2021)
- [Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning](https://mlsanity.com/paper/2103.15454/) (2021)

## Abstract (arXiv)

Distance metric learning (DML) is to learn the embeddings where examples from the same class are closer than examples from different classes. It can be cast as an optimization problem with triplet constraints. Due to the vast number of triplet constraints, a sampling strategy is essential for DML. With the tremendous success of deep learning in classifications, it has been applied for DML. When learning embeddings with deep neural networks (DNNs), only a mini-batch of data is available at each iteration. The set of triplet constraints has to be sampled within the mini-batch. Since a mini-batch cannot capture the neighbors in the original set well, it makes the learned embeddings sub-optimal. On the contrary, optimizing SoftMax loss, which is a classification loss, with DNN shows a superior performance in certain DML tasks. It inspires us to investigate the formulation of SoftMax. Our analysis shows that SoftMax loss is equivalent to a smoothed triplet loss where each class has a single center. In real-world data, one class can contain several local clusters rather than a single one, e.g., birds of different poses. Therefore, we propose the SoftTriple loss to extend the SoftMax loss with multiple centers for each class. Compared with conventional deep metric learning algorithms, optimizing SoftTriple loss can learn the embeddings without the sampling phase by mildly increasing the size of the last fully connected layer. Experiments on the benchmark fine-grained data sets demonstrate the effectiveness of the proposed loss function. Code is available at https://github.com/idstcv/SoftTriple