# Self-supervised Auxiliary Loss for Metric Learning in Music Similarity-based Retrieval and Auto-tagging

arXiv: https://arxiv.org/abs/2304.07449 · HTML: https://mlsanity.com/paper/2304.07449/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 15 references that are in the map.
- First posted on arXiv 15 Apr 2023.

## Builds on (most cited first, 10 of 10)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [SoftTriple Loss: Deep Metric Learning Without Triplet Sampling](https://mlsanity.com/paper/1909.05235/) (2019)
- [Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2103.09410/) (2021)
- [CDPAM: Contrastive learning for perceptual audio similarity](https://mlsanity.com/paper/2102.05109/) (2021)
- [Evaluation of CNN-based Automatic Music Tagging Models](https://mlsanity.com/paper/2006.00751/) (2020)
- [Semi-Supervised Music Tagging Transformer](https://mlsanity.com/paper/2111.13457/) (2021)
- [Metric Learning vs Classification for Disentangled Music Representation Learning](https://mlsanity.com/paper/2008.03729/) (2020)
- [SLADE: A Self-Training Framework For Distance Metric Learning](https://mlsanity.com/paper/2011.10269/) (2020)
- [Musical Audio Similarity with Self-supervised Convolutional Neural Networks](https://mlsanity.com/paper/2202.02112/) (2022)

## Abstract (arXiv)

In the realm of music information retrieval, similarity-based retrieval and auto-tagging serve as essential components. Given the limitations and non-scalability of human supervision signals, it becomes crucial for models to learn from alternative sources to enhance their performance. Self-supervised learning, which exclusively relies on learning signals derived from music audio data, has demonstrated its efficacy in the context of auto-tagging. In this study, we propose a model that builds on the self-supervised learning approach to address the similarity-based retrieval challenge by introducing our method of metric learning with a self-supervised auxiliary loss. Furthermore, diverging from conventional self-supervised learning methodologies, we discovered the advantages of concurrently training the model with both self-supervision and supervision signals, without freezing pre-trained models. We also found that refraining from employing augmentation during the fine-tuning phase yields better results. Our experimental results confirm that the proposed methodology enhances retrieval and tagging performance metrics in two distinct scenarios: one where human-annotated tags are consistently available for all music tracks, and another where such tags are accessible only for a subset of tracks.