# An Experimental Comparison Of Multi-view Self-supervised Methods For Music Tagging

arXiv: https://arxiv.org/abs/2404.09177 · HTML: https://mlsanity.com/paper/2404.09177/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 14 Apr 2024.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021)
- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [How Well Do Self-Supervised Models Transfer?](https://mlsanity.com/paper/2011.13377/) (2020)
- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [Contrastive Learning of General-Purpose Audio Representations](https://mlsanity.com/paper/2010.10915/) (2020)
- [VICRegL: Self-Supervised Learning of Local Visual Features](https://mlsanity.com/paper/2210.01571/) (2022)
- [BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation](https://mlsanity.com/paper/2103.06695/) (2021)
- [Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2103.09410/) (2021)
- [Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision](https://mlsanity.com/paper/2202.08360/) (2022)
- [Towards Learning Universal Audio Representations](https://mlsanity.com/paper/2111.12124/) (2021)
- [Supervised and Unsupervised Learning of Audio Representations for Music Understanding](https://mlsanity.com/paper/2210.03799/) (2022)
- [Contrastive Audio-Language Learning for Music](https://mlsanity.com/paper/2208.12208/) (2022)
- [vocadito: A dataset of solo vocals with f0, note, and lyric annotations](https://mlsanity.com/paper/2110.05580/) (2021)

## Built on by (most cited first, 4 of 4)

- [Just Ask for Music (JAM): Multimodal and Personalized Natural Language Music Recommendation](https://mlsanity.com/paper/2507.15826/) (2025)
- [From Real to Cloned Singer Identification](https://mlsanity.com/paper/2407.08647/) (2024)
- [Emergent musical properties of a transformer under contrastive self-supervised learning](https://mlsanity.com/paper/2506.23873/) (2025)
- [CoJEPA: Combining Contrastive Learning and JEPA for Global-Local Music Representations](https://mlsanity.com/paper/2608.30974/) (2026)

## Abstract (arXiv)

Self-supervised learning has emerged as a powerful way to pre-train generalizable machine learning models on large amounts of unlabeled data. It is particularly compelling in the music domain, where obtaining labeled data is time-consuming, error-prone, and ambiguous. During the self-supervised process, models are trained on pretext tasks, with the primary objective of acquiring robust and informative features that can later be fine-tuned for specific downstream tasks. The choice of the pretext task is critical as it guides the model to shape the feature space with meaningful constraints for information encoding. In the context of music, most works have relied on contrastive learning or masking techniques. In this study, we expand the scope of pretext tasks applied to music by investigating and comparing the performance of new self-supervised methods for music tagging. We open-source a simple ResNet model trained on a diverse catalog of millions of tracks. Our results demonstrate that, although most of these pre-training methods result in similar downstream results, contrastive learning consistently results in better downstream performance compared to other self-supervised pre-training methods. This holds true in a limited-data downstream context.