# MARBLE: Music Audio Representation Benchmark for Universal Evaluation

arXiv: https://arxiv.org/abs/2306.10548 · HTML: https://mlsanity.com/paper/2306.10548/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 71 references that are in the map.
- First posted on arXiv 18 Jun 2023.

## Builds on (most cited first, 30 of 42)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders](https://mlsanity.com/paper/1704.01279/) (2017)
- [Scaling and Benchmarking Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/1905.01235/) (2019)
- [Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders](https://mlsanity.com/paper/1910.12638/) (2019)
- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [Contrastive Learning of General-Purpose Audio Representations](https://mlsanity.com/paper/2010.10915/) (2020)
- [Hybrid Transformers for Music Source Separation](https://mlsanity.com/paper/2211.08553/) (2022)
- [MuLan: A Joint Embedding of Music Audio and Natural Language](https://mlsanity.com/paper/2208.12415/) (2022)
- [Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2103.09410/) (2021)
- [How Useful is Self-Supervised Pretraining for Visual Tasks?](https://mlsanity.com/paper/2003.14323/) (2020)
- [HEAR: Holistic Evaluation of Audio Representations](https://mlsanity.com/paper/2203.03022/) (2022)
- [Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognition](https://mlsanity.com/paper/1912.01679/) (2019)
- [madmom: a new Python Audio and Music Signal Processing Library](https://mlsanity.com/paper/1605.07008/) (2016)
- [Codified audio language modeling learns useful representations for music information retrieval](https://mlsanity.com/paper/2107.05677/) (2021)
- [Towards Learning Universal Audio Representations](https://mlsanity.com/paper/2111.12124/) (2021)
- [musicnn: Pre-trained convolutional neural networks for music audio tagging](https://mlsanity.com/paper/1909.06654/) (2019)
- [Music Demixing Challenge 2021](https://mlsanity.com/paper/2108.13559/) (2021)
- [Supervised and Unsupervised Learning of Audio Representations for Music Understanding](https://mlsanity.com/paper/2210.03799/) (2022)
- [MAP-Music2Vec: A Simple and Effective Baseline for Self-Supervised Music Audio Representation Learning](https://mlsanity.com/paper/2212.02508/) (2022)

## Built on by (most cited first, 30 of 38)

- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2410.19168/) (2024)
- [ChatMusician: Understanding and Generating Music Intrinsically with LLM](https://mlsanity.com/paper/2402.16153/) (2024)
- [MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization](https://mlsanity.com/paper/2501.01108/) (2025)
- [MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Models](https://mlsanity.com/paper/2408.01337/) (2024)
- [Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2505.15957/) (2025)
- [Foundation Models for Music: A Survey](https://mlsanity.com/paper/2408.14340/) (2024)
- [AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Models](https://mlsanity.com/paper/2506.05140/) (2025)
- [CMI-Bench: A Comprehensive Benchmark for Evaluating Music Instruction Following](https://mlsanity.com/paper/2506.12285/) (2025)
- [MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Music](https://mlsanity.com/paper/2402.09871/) (2024)
- [The Music Maestro or The Musically Challenged, A Massive Music Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2406.15885/) (2024)
- [Spiking Tucker Fusion Transformer for Audio-Visual Zero-Shot Learning](https://mlsanity.com/paper/2407.08130/) (2024)
- [OpenMU: Your Swiss Army Knife for Music Understanding](https://mlsanity.com/paper/2410.15573/) (2024)
- [Stem-JEPA: A Joint-Embedding Predictive Architecture for Musical Stem Compatibility Estimation](https://mlsanity.com/paper/2408.02514/) (2024)
- [Music Foundation Model as Generic Booster for Music Downstream Tasks](https://mlsanity.com/paper/2411.01135/) (2024)
- [MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning](https://mlsanity.com/paper/2310.09853/) (2023)
- [A Comprehensive Survey on Generative AI for Video-to-Music Generation](https://mlsanity.com/paper/2502.12489/) (2025)
- [Layer-wise Investigation of Large-Scale Self-Supervised Music Representation Models](https://mlsanity.com/paper/2505.16306/) (2025)
- [Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation](https://mlsanity.com/paper/2511.16757/) (2025)
- [Universal Music Representations? Evaluating Foundation Models on World Music Corpora](https://mlsanity.com/paper/2506.17055/) (2025)
- [A correlation-permutation approach for speech-music encoders model merging](https://mlsanity.com/paper/2506.11403/) (2025)
- [CultureMERT: Continual Pre-Training for Cross-Cultural Music Representation Learning](https://mlsanity.com/paper/2506.17818/) (2025)
- [Music Auto-Tagging with Robust Music Representation Learned via Domain Adversarial Training](https://mlsanity.com/paper/2401.15323/) (2024)
- [Prevailing Research Areas for Music AI in the Era of Foundation Models](https://mlsanity.com/paper/2409.09378/) (2024)
- [Keep what you need : extracting efficient subnetworks from large audio representation models](https://mlsanity.com/paper/2502.12925/) (2025)
- [USAD 2.0: Scaling Representation Distillation for Universal Audio Understanding](https://mlsanity.com/paper/2606.06444/) (2026)
- [Music Genre Classification using Large Language Models](https://mlsanity.com/paper/2410.08321/) (2024)
- [Zero-shot Musical Stem Retrieval with Joint-Embedding Predictive Architectures](https://mlsanity.com/paper/2411.19806/) (2024)
- [AudioCodecBench: A Comprehensive Benchmark for Audio Codec Evaluation](https://mlsanity.com/paper/2509.02349/) (2025)
- [Transcribing Rhythmic Patterns of the Guitar Track in Polyphonic Music](https://mlsanity.com/paper/2510.05756/) (2025)

## Abstract (arXiv)

In the era of extensive intersection between art and Artificial Intelligence (AI), such as image generation and fiction co-creation, AI for music remains relatively nascent, particularly in music understanding. This is evident in the limited work on deep music representations, the scarcity of large-scale datasets, and the absence of a universal and community-driven benchmark. To address this issue, we introduce the Music Audio Representation Benchmark for universaL Evaluation, termed MARBLE. It aims to provide a benchmark for various Music Information Retrieval (MIR) tasks by defining a comprehensive taxonomy with four hierarchy levels, including acoustic, performance, score, and high-level description. We then establish a unified protocol based on 14 tasks on 8 public-available datasets, providing a fair and standard assessment of representations of all open-sourced pre-trained models developed on music recordings as baselines. Besides, MARBLE offers an easy-to-use, extendable, and reproducible suite for the community, with a clear statement on copyright issues on datasets. Results suggest recently proposed large-scale pre-trained musical language models perform the best in most tasks, with room for further improvement. The leaderboard and toolkit repository are published at https://marble-bm.shef.ac.uk to promote future music AI research.