# Self-Similarity-Based and Novelty-based loss for music structure analysis

arXiv: https://arxiv.org/abs/2309.02243 · HTML: https://mlsanity.com/paper/2309.02243/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 42 references that are in the map.
- First posted on arXiv 5 Sep 2023.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [Unsupervised Learning of Deep Features for Music Segmentation](https://mlsanity.com/paper/2108.12955/) (2021)
- [To catch a chorus, verse, intro, or anything else: Analyzing a song with structural functions](https://mlsanity.com/paper/2205.14700/) (2022)
- [Supervised Metric Learning for Music Structure Features](https://mlsanity.com/paper/2110.09000/) (2021)
- [SSM-Net: feature learning for Music Structure Analysis using a Self-Similarity-Matrix based loss](https://mlsanity.com/paper/2211.08141/) (2022)

## Built on by (most cited first, 3 of 3)

- [ImprovNet -- Generating Controllable Musical Improvisations with Iterative Corruption Refinement](https://mlsanity.com/paper/2502.04522/) (2025)
- [V2M-Zero: Zero-Pair Time-Aligned Video-to-Music Generation](https://mlsanity.com/paper/2603.11042/) (2026)
- [Do Foundational Audio Encoders Understand Music Structure?](https://mlsanity.com/paper/2512.17209/) (2025)

## Abstract (arXiv)

Music Structure Analysis (MSA) is the task aiming at identifying musical segments that compose a music track and possibly label them based on their similarity. In this paper we propose a supervised approach for the task of music boundary detection. In our approach we simultaneously learn features and convolution kernels. For this we jointly optimize -- a loss based on the Self-Similarity-Matrix (SSM) obtained with the learned features, denoted by SSM-loss, and -- a loss based on the novelty score obtained applying the learned kernels to the estimated SSM, denoted by novelty-loss. We also demonstrate that relative feature learning, through self-attention, is beneficial for the task of MSA. Finally, we compare the performances of our approach to previously proposed approaches on the standard RWC-Pop, and various subsets of SALAMI.