mlsanityData to 7 Oct 2026

Towards Proper Contrastive Self-supervised Learning Strategies For Music Audio Representation

arXiv 2207.04471 ↗First posted 10 Jul 2022cs.SD cs.AI cs.MMcited by 5

Not on the map. Why?

Abstract (arXiv)

The common research goal of self-supervised learning is to extract a general representation which an arbitrary downstream task would benefit from. In this work, we investigate music audio representation learned from different contrastive self-supervised learning schemes and empirically evaluate the embedded vectors on various music information retrieval (MIR) tasks where different levels of the music perception are concerned. We analyze the results to discuss the proper direction of contrastive learning strategies for different MIR tasks. We show that these representations convey a comprehensive information about the auditory characteristics of music in general, although each of the self-supervision strategies has its own effectiveness in certain aspect of information.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 34 of its 43 references that are in the map. First posted on arXiv 10 Jul 2022.

Builds on

34 of 43 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  6. 2018Deep contextualized word representationscited by 2,285
  7. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  8. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  9. 2016Colorful Image Colorizationcited by 1,113
  10. 2017Neural Collaborative Filteringcited by 1,108
  11. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  12. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
22 more
  1. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
  2. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  3. 2016FMA: A Dataset For Music Analysiscited by 205
  4. 2019Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approachescited by 181
  5. 2020Contrastive Learning of General-Purpose Audio Representationscited by 168
  6. 2020Multi-task self-supervised learning for Robust Speech Recognitioncited by 127
  7. 2019Learning Problem-agnostic Speech Representations from Multiple Self-supervised Taskscited by 87
  8. 2019Generative Pre-Training for Speech with Autoregressive Predictive Codingcited by 82
  9. 2021Contrastive Learning of Musical Representationscited by 81
  10. 2020Data Augmenting Contrastive Learning of Speech Representations in the Time Domaincited by 63
  11. 2020Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio Representationcited by 58
  12. 2016Convolutional Recurrent Neural Networks for Music Classificationcited by 49
  13. 2019musicnn: Pre-trained convolutional neural networks for music audio taggingcited by 49
  14. 2017Unsupervised Learning of Semantic Audio Representationscited by 44
  15. 2016Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencodercited by 34
  16. 2020CLAR: Contrastive Learning of Auditory Representationscited by 31
  17. 2021Semi-Supervised Music Tagging Transformercited by 24
  18. 2020Disentangled Multidimensional Metric Learning for Music Similaritycited by 16
  19. 2017Multi-Level and Multi-Scale Feature Aggregation Using Pre-trained Convolutional Neural Networks for Music Auto-taggingcited by 8
  20. 2018One Deep Music Representation to Rule Them All? : A comparative analysis of different representation learning strategiescited by 7
  21. 2016Song Recommendation with Non-Negative Matrix Factorization and Graph Total Variationcited by 4
  22. 2020Learning to rank music tracks using triplet losscited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 2.

5 papers.

  1. 2025Scaling Self-Supervised Representation Learning for Symbolic Piano Performancecited by 7
  2. 2024On the Effect of Data-Augmentation on Local Embedding Properties in the Contrastive Learning of Music Audio Representationscited by 4
  3. 2024I can listen but cannot read: An evaluation of two-tower multimodal systems for instrument recognitioncited by 4
  4. 2023Unsupervised Musical Object Discovery from Audiocited by 2
  5. 2025Emergent musical properties of a transformer under contrastive self-supervised learningcited by 2