# SLADE: A Self-Training Framework For Distance Metric Learning

arXiv: https://arxiv.org/abs/2011.10269 · HTML: https://mlsanity.com/paper/2011.10269/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2020.

## Built on by (most cited first, 8 of 8)

- [Multi-modal Alignment using Representation Codebook](https://mlsanity.com/paper/2203.00048/) (2022)
- [Confidence May Cheat: Self-Training on Graph Neural Networks under Distribution Shift](https://mlsanity.com/paper/2201.11349/) (2022)
- [DQS3D: Densely-matched Quantization-aware Semi-supervised 3D Detection](https://mlsanity.com/paper/2304.13031/) (2023)
- [Self-Taught Metric Learning without Labels](https://mlsanity.com/paper/2205.01903/) (2022)
- [Bridging Gap between Image Pixels and Semantics via Supervision: A Survey](https://mlsanity.com/paper/2107.13757/) (2021)
- [Learning Rich Nearest Neighbor Representations from Self-supervised Ensembles](https://mlsanity.com/paper/2110.10293/) (2021)
- [Augmenting Vision Language Pretraining by Learning Codebook with Visual Semantics](https://mlsanity.com/paper/2208.00475/) (2022)
- [Self-supervised Auxiliary Loss for Metric Learning in Music Similarity-based Retrieval and Auto-tagging](https://mlsanity.com/paper/2304.07449/) (2023)

## Abstract (arXiv)

Most existing distance metric learning approaches use fully labeled data to learn the sample similarities in an embedding space. We present a self-training framework, SLADE, to improve retrieval performance by leveraging additional unlabeled data. We first train a teacher model on the labeled data and use it to generate pseudo labels for the unlabeled data. We then train a student model on both labels and pseudo labels to generate final feature embeddings. We use self-supervised representation learning to initialize the teacher model. To better deal with noisy pseudo labels generated by the teacher network, we design a new feature basis learning component for the student network, which learns basis functions of feature representations for unlabeled data. The learned basis vectors better measure the pairwise similarity and are used to select high-confident samples for training the student network. We evaluate our method on standard retrieval benchmarks: CUB-200, Cars-196 and In-shop. Experimental results demonstrate that our approach significantly improves the performance over the state-of-the-art methods.