mlsanityData to 14 Oct 2026

MARBLE: Music Audio Representation Benchmark for Universal Evaluation

arXiv 2306.10548 ↗First posted 18 Jun 2023cs.SD cs.AI cs.LGcited by 38

Not on the map. Why?

Abstract (arXiv)

In the era of extensive intersection between art and Artificial Intelligence (AI), such as image generation and fiction co-creation, AI for music remains relatively nascent, particularly in music understanding. This is evident in the limited work on deep music representations, the scarcity of large-scale datasets, and the absence of a universal and community-driven benchmark. To address this issue, we introduce the Music Audio Representation Benchmark for universaL Evaluation, termed MARBLE. It aims to provide a benchmark for various Music Information Retrieval (MIR) tasks by defining a comprehensive taxonomy with four hierarchy levels, including acoustic, performance, score, and high-level description. We then establish a unified protocol based on 14 tasks on 8 public-available datasets, providing a fair and standard assessment of representations of all open-sourced pre-trained models developed on music recordings as baselines. Besides, MARBLE offers an easy-to-use, extendable, and reproducible suite for the community, with a clear statement on copyright issues on datasets. Results suggest recently proposed large-scale pre-trained musical language models perform the best in most tasks, with room for further improvement. The leaderboard and toolkit repository are published at https://marble-bm.shef.ac.uk to promote future music AI research.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). Builds on 42 of its 71 references that are in the map. First posted on arXiv 18 Jun 2023.

Builds on

42 of 71 references.

References · 42

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  6. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  7. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  8. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  9. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
  10. 2020Jukebox: A Generative Model for Musiccited by 542
  11. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 530
  12. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 406
30 more
  1. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 225
  2. 2019Scaling and Benchmarking Self-Supervised Visual Representation Learningcited by 201
  3. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 198
  4. 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 181
  5. 2020Contrastive Learning of General-Purpose Audio Representationscited by 169
  6. 2022Hybrid Transformers for Music Source Separationcited by 134
  7. 2022MuLan: A Joint Embedding of Music Audio and Natural Languagecited by 87
  8. 2021Contrastive Learning of Musical Representationscited by 82
  9. 2020How Useful is Self-Supervised Pretraining for Visual Tasks?cited by 80
  10. 2022HEAR: Holistic Evaluation of Audio Representationscited by 78
  11. 2019Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognitioncited by 66
  12. 2016madmom: a new Python Audio and Music Signal Processing Librarycited by 64
  13. 2021Codified audio language modeling learns useful representations for music information retrievalcited by 64
  14. 2021Towards Learning Universal Audio Representationscited by 51
  15. 2019musicnn: Pre-trained convolutional neural networks for music audio taggingcited by 50
  16. 2021Music Demixing Challenge 2021cited by 38
  17. 2022Supervised and Unsupervised Learning of Audio Representations for Music Understandingcited by 35
  18. 2022MAP-Music2Vec: A Simple and Effective Baseline for Self-Supervised Music Audio Representation Learningcited by 22
  19. 2015Deep Learning and Music Adversariescited by 20
  20. 2021Multi-Task Self-Supervised Pre-Training for Music Classificationcited by 18
  21. 2018Music Style Transfer: A Position Papercited by 16
  22. 2017End-to-End Musical Key Estimation Using a Convolutional Neural Networkcited by 13
  23. 2021BeatNet: CRNN and Particle Filtering for Online Joint Beat Downbeat and Meter Trackingcited by 13
  24. 2021Learning music audio representations via weak language supervisioncited by 13
  25. 2022Transfer Learning of wav2vec 2.0 for Automatic Lyric Transcriptioncited by 11
  26. 2023Contrastive Learning-Based Audio to Lyrics Alignment for Multiple Languagescited by 11
  27. 2022Contrastive Learning with Positive-Negative Frame Mask for Music Representationcited by 10
  28. 2022Deformable CNN and Imbalance-Aware Feature Learning for Singing Technique Classificationcited by 8
  29. 2022PoLyScriber: Integrated Fine-tuning of Extractor and Lyrics Transcriber for Polyphonic Musiccited by 7
  30. 2021Transfer Learning with Jukebox for Music Source Separationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 14; 2025: 15; 2026: 8.

38 papers.

Sort:

By area:

  1. 2025Qwen3-Omni Technical Reportcited by 497
  2. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
  3. 2024ChatMusician: Understanding and Generating Music Intrinsically with LLMcited by 68
  4. 2025MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantizationcited by 68
  5. 2024MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Modelscited by 44
  6. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 39
  7. 2024Foundation Models for Music: A Surveycited by 37
  8. 2025AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Modelscited by 18
  9. 2025CMI-Bench: A Comprehensive Benchmark for Evaluating Music Instruction Followingcited by 17
  10. 2024MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Musiccited by 16
  11. 2024The Music Maestro or The Musically Challenged, A Massive Music Evaluation Benchmark for Large Language Modelscited by 10
  12. 2024Spiking Tucker Fusion Transformer for Audio-Visual Zero-Shot Learningcited by 10
26 more
  1. 2024OpenMU: Your Swiss Army Knife for Music Understandingcited by 9
  2. 2024Stem-JEPA: A Joint-Embedding Predictive Architecture for Musical Stem Compatibility Estimationcited by 8
  3. 2024Music Foundation Model as Generic Booster for Music Downstream Taskscited by 6
  4. 2023MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuningcited by 4
  5. 2025A Comprehensive Survey on Generative AI for Video-to-Music Generationcited by 4
  6. 2025Layer-wise Investigation of Large-Scale Self-Supervised Music Representation Modelscited by 4
  7. 2025Revisiting Audio-language Pretraining for Learning General-purpose Audio Representationcited by 4
  8. 2025Universal Music Representations? Evaluating Foundation Models on World Music Corporacited by 3
  9. 2025A correlation-permutation approach for speech-music encoders model mergingcited by 2
  10. 2025CultureMERT: Continual Pre-Training for Cross-Cultural Music Representation Learningcited by 2
  11. 2024Music Auto-Tagging with Robust Music Representation Learned via Domain Adversarial Trainingcited by 1
  12. 2024Prevailing Research Areas for Music AI in the Era of Foundation Modelscited by 1
  13. 2025Keep what you need : extracting efficient subnetworks from large audio representation modelscited by 1
  14. 2026USAD 2.0: Scaling Representation Distillation for Universal Audio Understandingcited by 1
  15. 2024Music Genre Classification using Large Language Modelscited by 0
  16. 2024Zero-shot Musical Stem Retrieval with Joint-Embedding Predictive Architecturescited by 0
  17. 2025AudioCodecBench: A Comprehensive Benchmark for Audio Codec Evaluationcited by 0
  18. 2025Transcribing Rhythmic Patterns of the Guitar Track in Polyphonic Musiccited by 0
  19. 2025Twenty-Five Years of MIR Research: Achievements, Practices, Evaluations, and Future Challengescited by 0
  20. 2026Expectation and Acoustic Neural Network Representations Enhance Music Identification from Brain Activitycited by 0
  21. 2026Persian MusicGen: A Large-Scale Dataset and Culturally-Aware Generative Model for Persian Musiccited by 0
  22. 2026Exploring LLMs for South Asian Music Understanding and Generationcited by 0
  23. 2026Listening Like a Judge: A Music-Aware Framework for Automatic Singing Performance Evaluationcited by 0
  24. 2026What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Modelscited by 0
  25. 2026Music Hallucination in Audio-Language Models: A Hierarchical Formulation and Empirical Studycited by 0
  26. 2026From Isolated Feature to Orbits: Discovering Music Concepts via Multi-SAE Alignmentcited by 0