# mir_ref: A Representation Evaluation Framework for Music Information Retrieval Tasks

arXiv: https://arxiv.org/abs/2312.05994 · HTML: https://mlsanity.com/paper/2312.05994/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2023.

## Built on by (most cited first, 6 of 6)

- [MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Models](https://mlsanity.com/paper/2408.01337/) (2024)
- [Foundation Models for Music: A Survey](https://mlsanity.com/paper/2408.14340/) (2024)
- [Music2Latent: Consistency Autoencoders for Latent Audio Compression](https://mlsanity.com/paper/2408.06500/) (2024)
- [Towards a Unified Representation Evaluation Framework Beyond Downstream Tasks](https://mlsanity.com/paper/2505.06224/) (2025)
- [Music2Latent2: Audio Compression with Summary Embeddings and Autoregressive Decoding](https://mlsanity.com/paper/2501.17578/) (2025)
- [Learning Music Audio Representations With Limited Data](https://mlsanity.com/paper/2505.06042/) (2025)

## Abstract (arXiv)

Music Information Retrieval (MIR) research is increasingly leveraging representation learning to obtain more compact, powerful music audio representations for various downstream MIR tasks. However, current representation evaluation methods are fragmented due to discrepancies in audio and label preprocessing, downstream model and metric implementations, data availability, and computational resources, often leading to inconsistent and limited results. In this work, we introduce mir_ref, an MIR Representation Evaluation Framework focused on seamless, transparent, local-first experiment orchestration to support representation development. It features implementations of a variety of components such as MIR datasets, tasks, embedding models, and tools for result analysis and visualization, while facilitating the implementation of custom components. To demonstrate its utility, we use it to conduct an extensive evaluation of several embedding models across various tasks and datasets, including evaluating their robustness to various audio perturbations and the ease of extracting relevant information from them.