# Unsupervised Learning of Deep Features for Music Segmentation

arXiv: https://arxiv.org/abs/2108.12955 · HTML: https://mlsanity.com/paper/2108.12955/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 33 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 4 of 4)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [Unsupervised Learning of Semantic Audio Representations](https://mlsanity.com/paper/1711.02209/) (2017)

## Built on by (most cited first, 12 of 12)

- [Supervised and Unsupervised Learning of Audio Representations for Music Understanding](https://mlsanity.com/paper/2210.03799/) (2022)
- [The Power of Fragmentation: A Hierarchical Transformer Model for Structural Segmentation in Symbolic Music Generation](https://mlsanity.com/paper/2205.08579/) (2022)
- [Pitchclass2vec: Symbolic Music Structure Segmentation with Chord Embeddings](https://mlsanity.com/paper/2303.15306/) (2023)
- [On the Effect of Data-Augmentation on Local Embedding Properties in the Contrastive Learning of Music Audio Representations](https://mlsanity.com/paper/2401.08889/) (2024)
- [Self-Similarity-Based and Novelty-based loss for music structure analysis](https://mlsanity.com/paper/2309.02243/) (2023)
- [SSM-Net: feature learning for Music Structure Analysis using a Self-Similarity-Matrix based loss](https://mlsanity.com/paper/2211.08141/) (2022)
- [Why Perturbing Symbolic Music is Necessary: Fitting the Distribution of Never-used Notes through a Joint Probabilistic Diffusion Model](https://mlsanity.com/paper/2408.01950/) (2024)
- [Exploring single-song autoencoding schemes for audio-based music structure analysis](https://mlsanity.com/paper/2110.14437/) (2021)
- [Barwise Compression Schemes for Audio-Based Music Structure Analysis](https://mlsanity.com/paper/2202.04981/) (2022)
- [Learning Hierarchical Metrical Structure Beyond Measures](https://mlsanity.com/paper/2209.10259/) (2022)
- [Barwise Section Boundary Detection in Symbolic Music Using Convolutional Neural Networks](https://mlsanity.com/paper/2509.16566/) (2025)
- [Unsupervised Evaluation of Deep Audio Embeddings for Music Structure Analysis](https://mlsanity.com/paper/2603.27218/) (2026)

## Abstract (arXiv)

Music segmentation refers to the dual problem of identifying boundaries between, and labeling, distinct music segments, e.g., the chorus, verse, bridge etc. in popular music. The performance of a range of music segmentation algorithms has been shown to be dependent on the audio features chosen to represent the audio. Some approaches have proposed learning feature transformations from music segment annotation data, although, such data is time consuming or expensive to create and as such these approaches are likely limited by the size of their datasets. While annotated music segmentation data is a scarce resource, the amount of available music audio is much greater. In the neighboring field of semantic audio unsupervised deep learning has shown promise in improving the performance of solutions to the query-by-example and sound classification tasks. In this work, unsupervised training of deep feature embeddings using convolutional neural networks (CNNs) is explored for music segmentation. The proposed techniques exploit only the time proximity of audio features that is implicit in any audio timeline. Employing these embeddings in a classic music segmentation algorithm is shown not only to significantly improve the performance of this algorithm, but obtain state of the art performance in unsupervised music segmentation.