# Modeling the Compatibility of Stem Tracks to Generate Music Mashups

arXiv: https://arxiv.org/abs/2103.14208 · HTML: https://mlsanity.com/paper/2103.14208/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro](https://mlsanity.com/paper/1701.07717/) (2017)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [Music Source Separation in the Waveform Domain](https://mlsanity.com/paper/1911.13254/) (2019)
- [madmom: a new Python Audio and Music Signal Processing Library](https://mlsanity.com/paper/1605.07008/) (2016)
- [The 2018 Signal Separation Evaluation Campaign](https://mlsanity.com/paper/1804.06267/) (2018)
- [Neural Loop Combiner: Neural Network Models for Assessing the Compatibility of Loops](https://mlsanity.com/paper/2008.02011/) (2020)
- [Singing voice separation: a study on training data](https://mlsanity.com/paper/1906.02618/) (2019)
- [Sampling strategies in Siamese Networks for unsupervised speech representation learning](https://mlsanity.com/paper/1804.11297/) (2018)
- [Jointly Detecting and Separating Singing Voice: A Multi-Task Approach](https://mlsanity.com/paper/1804.01650/) (2018)

## Built on by (most cited first, 5 of 5)

- [COCOLA: Coherence-Oriented Contrastive Learning of Musical Audio Representations](https://mlsanity.com/paper/2404.16969/) (2024)
- [SampleMatch: Drum Sample Retrieval by Musical Context](https://mlsanity.com/paper/2208.01141/) (2022)
- [Stem-JEPA: A Joint-Embedding Predictive Architecture for Musical Stem Compatibility Estimation](https://mlsanity.com/paper/2408.02514/) (2024)
- [A Comprehensive Survey for Evaluation Methodologies of AI-Generated Music](https://mlsanity.com/paper/2308.13736/) (2023)
- [Multi-Genre Music Transformer -- Composing Full Length Musical Piece](https://mlsanity.com/paper/2301.02385/) (2023)

## Abstract (arXiv)

A music mashup combines audio elements from two or more songs to create a new work. To reduce the time and effort required to make them, researchers have developed algorithms that predict the compatibility of audio elements. Prior work has focused on mixing unaltered excerpts, but advances in source separation enable the creation of mashups from isolated stems (e.g., vocals, drums, bass, etc.). In this work, we take advantage of separated stems not just for creating mashups, but for training a model that predicts the mutual compatibility of groups of excerpts, using self-supervised and semi-supervised methods. Specifically, we first produce a random mashup creation pipeline that combines stem tracks obtained via source separation, with key and tempo automatically adjusted to match, since these are prerequisites for high-quality mashups. To train a model to predict compatibility, we use stem tracks obtained from the same song as positive examples, and random combinations of stems with key and/or tempo unadjusted as negative examples. To improve the model and use more data, we also train on "average" examples: random combinations with matching key and tempo, where we treat them as unlabeled data as their true compatibility is unknown. To determine whether the combined signal or the set of stem signals is more indicative of the quality of the result, we experiment on two model architectures and train them using semi-supervised learning technique. Finally, we conduct objective and subjective evaluations of the system, comparing them to a standard rule-based system.