# Multi-Source Contrastive Learning from Musical Audio

arXiv: https://arxiv.org/abs/2302.07077 · HTML: https://mlsanity.com/paper/2302.07077/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2023.

## Built on by (most cited first, 4 of 4)

- [COCOLA: Coherence-Oriented Contrastive Learning of Musical Audio Representations](https://mlsanity.com/paper/2404.16969/) (2024)
- [Stem-JEPA: A Joint-Embedding Predictive Architecture for Musical Stem Compatibility Estimation](https://mlsanity.com/paper/2408.02514/) (2024)
- [Emergent musical properties of a transformer under contrastive self-supervised learning](https://mlsanity.com/paper/2506.23873/) (2025)
- [Contrastive timbre representations for musical instrument and synthesizer retrieval](https://mlsanity.com/paper/2509.13285/) (2025)

## Abstract (arXiv)

Contrastive learning constitutes an emerging branch of self-supervised learning that leverages large amounts of unlabeled data, by learning a latent space, where pairs of different views of the same sample are associated. In this paper, we propose musical source association as a pair generation strategy in the context of contrastive music representation learning. To this end, we modify COLA, a widely used contrastive learning audio framework, to learn to associate a song excerpt with a stochastically selected and automatically extracted vocal or instrumental source. We further introduce a novel modification to the contrastive loss to incorporate information about the existence or absence of specific sources. Our experimental evaluation in three different downstream tasks (music auto-tagging, instrument classification and music genre classification) using the publicly available Magna-Tag-A-Tune (MTAT) as a source dataset yields competitive results to existing literature methods, as well as faster network convergence. The results also show that this pre-training method can be steered towards specific features, according to the selected musical source, while also being dependent on the quality of the separated sources.