mlsanityData to 14 Oct 2026

Learning Music Sequence Representation from Text Supervision

arXiv 2305.19602 ↗First posted 31 May 2023cs.SD cs.LG eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

Music representation learning is notoriously difficult for its complex human-related concepts contained in the sequence of numerical signals. To excavate better MUsic SEquence Representation from labeled audio, we propose a novel text-supervision pre-training method, namely MUSER. MUSER adopts an audio-spectrum-text tri-modal contrastive learning framework, where the text input could be any form of meta-data with the help of text templates while the spectrum is derived from an audio sequence. Our experiments reveal that MUSER could be more flexibly adapted to downstream tasks compared with the current data-hungry pre-training method, and it only requires 0.056% of pre-training data to achieve the state-of-the-art performance.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 17 of its 24 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

17 of 24 references.

References · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2017Non-local Neural Networkscited by 2,293
  5. 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
  6. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 339
  7. 2021Contrastive Learning of Musical Representationscited by 82
  8. 2021Codified audio language modeling learns useful representations for music information retrievalcited by 64
  9. 2019musicnn: Pre-trained convolutional neural networks for music audio taggingcited by 50
  10. 2021A Simple Long-Tailed Recognition Baseline via Vision-Language Modelcited by 49
  11. 2016Automatic tagging using deep convolutional neural networkscited by 40
  12. 2019Toward Interpretable Music Tagging with Self-Attentioncited by 30
5 more
  1. 2017End-to-end learning for music audio tagging at scalecited by 25
  2. 2021ESResNe(X)t-fbsp: Learning Robust Time-Frequency Transformation of Audiocited by 24
  3. 2015Deep Learning and Music Adversariescited by 20
  4. 2018Music Genre Classification using Machine Learning Techniquescited by 7
  5. 2019Representation Learning of Music Using Artist, Album, and Track Informationcited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 3.

5 papers.

  1. 2024Heterogeneous Contrastive Learning for Foundation Models and Beyondcited by 27
  2. 2024Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompanimentcited by 13
  3. 2024Augment, Drop & Swap: Improving Diversity in LLM Captions for Efficient Music-Text Representation Learningcited by 3
  4. 2023Unsupervised Musical Object Discovery from Audiocited by 2
  5. 2023MDSC: Towards Evaluating the Style Consistency Between Music and Dancecited by 1