# Multi-level Distance Regularization for Deep Metric Learning

arXiv: https://arxiv.org/abs/2102.04223 · HTML: https://mlsanity.com/paper/2102.04223/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 8 Feb 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning](https://mlsanity.com/paper/1904.06627/) (2019)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [NormFace: L2 Hypersphere Embedding for Face Verification](https://mlsanity.com/paper/1704.06369/) (2017)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [SoftTriple Loss: Deep Metric Learning Without Triplet Sampling](https://mlsanity.com/paper/1909.05235/) (2019)
- [Cross-Batch Memory for Embedding Learning](https://mlsanity.com/paper/1912.06798/) (2019)
- [Deep Metric Learning with Hierarchical Triplet Loss](https://mlsanity.com/paper/1810.06951/) (2018)
- [Ranked List Loss for Deep Metric Learning](https://mlsanity.com/paper/1903.03238/) (2019)
- [Attention-based Ensemble for Deep Metric Learning](https://mlsanity.com/paper/1804.00382/) (2018)
- [Learning Deep Embeddings with Histogram Loss](https://mlsanity.com/paper/1611.00822/) (2016)
- [Hard-Aware Deeply Cascaded Embedding](https://mlsanity.com/paper/1611.05720/) (2016)
- [Deep Metric Learning with BIER: Boosting Independent Embeddings Robustly](https://mlsanity.com/paper/1801.04815/) (2018)
- [DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning](https://mlsanity.com/paper/2004.13458/) (2020)
- [Divide and Conquer the Embedding Space for Metric Learning](https://mlsanity.com/paper/1906.05990/) (2019)
- [Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions](https://mlsanity.com/paper/1512.09272/) (2015)
- [Metric Learning With HORDE: High-Order Regularizer for Deep Embeddings](https://mlsanity.com/paper/1908.02735/) (2019)
- [PADS: Policy-Adapted Sampling for Visual Similarity Learning](https://mlsanity.com/paper/2003.11113/) (2020)

## Built on by (most cited first, 6 of 6)

- [Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework](https://mlsanity.com/paper/2204.13207/) (2022)
- [Rethinking the Optimization of Average Precision: Only Penalizing Negative Instances before Positive Ones is Enough](https://mlsanity.com/paper/2102.04640/) (2021)
- [Deep Metric Learning with Chance Constraints](https://mlsanity.com/paper/2209.09060/) (2022)
- [Generalized Sum Pooling for Metric Learning](https://mlsanity.com/paper/2308.09228/) (2023)
- [Lifestyle-Informed Personalized Blood Biomarker Prediction via Novel Representation Learning](https://mlsanity.com/paper/2407.07277/) (2024)
- [No Pairs Left Behind: Improving Metric Learning with Regularized Triplet Objective](https://mlsanity.com/paper/2210.09506/) (2022)

## Abstract (arXiv)

We propose a novel distance-based regularization method for deep metric learning called Multi-level Distance Regularization (MDR). MDR explicitly disturbs a learning procedure by regularizing pairwise distances between embedding vectors into multiple levels that represents a degree of similarity between a pair. In the training stage, the model is trained with both MDR and an existing loss function of deep metric learning, simultaneously; the two losses interfere with the objective of each other, and it makes the learning process difficult. Moreover, MDR prevents some examples from being ignored or overly influenced in the learning process. These allow the parameters of the embedding network to be settle on a local optima with better generalization. Without bells and whistles, MDR with simple Triplet loss achieves the-state-of-the-art performance in various benchmark datasets: CUB-200-2011, Cars-196, Stanford Online Products, and In-Shop Clothes Retrieval. We extensively perform ablation studies on its behaviors to show the effectiveness of MDR. By easily adopting our MDR, the previous approaches can be improved in performance and generalization ability.