mlsanityData to 14 Oct 2026

Layer-wise Investigation of Large-Scale Self-Supervised Music Representation Models

arXiv 2505.16306 ↗First posted 22 May 2025cs.SD cs.AI eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

Recently, pre-trained models for music information retrieval based on self-supervised learning (SSL) are becoming popular, showing success in various downstream tasks. However, there is limited research on the specific meanings of the encoded information and their applicability. Exploring these aspects can help us better understand their capabilities and limitations, leading to more effective use in downstream tasks. In this study, we analyze the advanced music representation model MusicFM and the newly emerged SSL model MuQ. We focus on three main aspects: (i) validating the advantages of SSL models across multiple downstream tasks, (ii) exploring the specialization of layer-wise information for different tasks, and (iii) comparing performance differences when selecting specific layers. Through this analysis, we reveal insights into the structure and potential applications of SSL models in music information retrieval.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

17 of 31 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2018Deep contextualized word representationscited by 2,290
  5. 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
  6. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  7. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 225
  8. 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 181
  9. 2022Self-supervised Learning with Random-projection Quantizer for Speech Recognitioncited by 162
  10. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
  11. 2021Contrastive Learning of Musical Representationscited by 82
  12. 2025MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantizationcited by 68
5 more
  1. 2021Codified audio language modeling learns useful representations for music information retrievalcited by 64
  2. 2023A Foundation Model for Music Informaticscited by 46
  3. 2023MARBLE: Music Audio Representation Benchmark for Universal Evaluationcited by 38
  4. 2022Supervised and Unsupervised Learning of Audio Representations for Music Understandingcited by 35
  5. 2022Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?cited by 27

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Sparse Autoencoders Make Audio Foundation Models more Explainablecited by 1
  2. 2026Revisiting Content-Based Music Recommendation: Efficient Feature Aggregation from Large-Scale Music Modelscited by 0
  3. 2026What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Modelscited by 0
  4. 2026Gugak-VocalSet: A Multi-Genre Studio-Quality Dataset of Korean Traditional Singing with Sigimsae Annotationscited by 0