# Evaluation of CNN-based Automatic Music Tagging Models

arXiv: https://arxiv.org/abs/2006.00751 · HTML: https://mlsanity.com/paper/2006.00751/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2020.

## Built on by (most cited first, 30 of 41)

- [MuLan: A Joint Embedding of Music Audio and Natural Language](https://mlsanity.com/paper/2208.12415/) (2022)
- [MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization](https://mlsanity.com/paper/2501.01108/) (2025)
- [Codified audio language modeling learns useful representations for music information retrieval](https://mlsanity.com/paper/2107.05677/) (2021)
- [MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response](https://mlsanity.com/paper/2309.08730/) (2023)
- [Supervised and Unsupervised Learning of Audio Representations for Music Understanding](https://mlsanity.com/paper/2210.03799/) (2022)
- [MusCaps: Generating Captions for Music Audio](https://mlsanity.com/paper/2104.11984/) (2021)
- [MuseChat: A Conversational Music Recommendation System for Videos](https://mlsanity.com/paper/2310.06282/) (2023)
- [Language-Guided Music Recommendation for Video via Prompt Analogies](https://mlsanity.com/paper/2306.09327/) (2023)
- [Low-Resource Music Genre Classification with Cross-Modal Neural Model Reprogramming](https://mlsanity.com/paper/2211.01317/) (2022)
- [SpecTNT: a Time-Frequency Transformer for Music Audio](https://mlsanity.com/paper/2110.09127/) (2021)
- [Learning music audio representations via weak language supervision](https://mlsanity.com/paper/2112.04214/) (2021)
- [Tr"aumerAI: Dreaming Music with StyleGAN](https://mlsanity.com/paper/2102.04680/) (2021)
- [The Chamber Ensemble Generator: Limitless High-Quality MIR Data via Generative Modeling](https://mlsanity.com/paper/2209.14458/) (2022)
- [OpenMU: Your Swiss Army Knife for Music Understanding](https://mlsanity.com/paper/2410.15573/) (2024)
- [YM2413-MDB: A Multi-Instrumental FM Video Game Music Dataset with Emotion Annotations](https://mlsanity.com/paper/2211.07131/) (2022)
- [Perceptual Musical Features for Interpretable Audio Tagging](https://mlsanity.com/paper/2312.11234/) (2023)
- [LC-Protonets: Multi-Label Few-Shot Learning for World Music Audio Tagging](https://mlsanity.com/paper/2409.11264/) (2024)
- [Jointist: Joint Learning for Multi-instrument Transcription and Its Applications](https://mlsanity.com/paper/2206.10805/) (2022)
- [Improving Sound Event Classification by Increasing Shift Invariance in Convolutional Neural Networks](https://mlsanity.com/paper/2107.00623/) (2021)
- [Audio Embeddings as Teachers for Music Classification](https://mlsanity.com/paper/2306.17424/) (2023)
- [From West to East: Who can understand the music of the others better?](https://mlsanity.com/paper/2307.09795/) (2023)
- [Joint Music and Language Attention Models for Zero-shot Music Tagging](https://mlsanity.com/paper/2310.10159/) (2023)
- [I can listen but cannot read: An evaluation of two-tower multimodal systems for instrument recognition](https://mlsanity.com/paper/2407.18058/) (2024)
- [Deep Learning Based EDM Subgenre Classification using Mel-Spectrogram and Tempogram Features](https://mlsanity.com/paper/2110.08862/) (2021)
- [Music-to-Text Synaesthesia: Generating Descriptive Text from Music Recordings](https://mlsanity.com/paper/2210.00434/) (2022)
- [Jointist: Simultaneous Improvement of Multi-instrument Transcription and Music Source Separation via Joint Training](https://mlsanity.com/paper/2302.00286/) (2023)
- [Myna: Masking-Based Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2502.12511/) (2025)
- [Universal Music Representations? Evaluating Foundation Models on World Music Corpora](https://mlsanity.com/paper/2506.17055/) (2025)
- [Listen, Read, and Identify: Multimodal Singing Language Identification of Music](https://mlsanity.com/paper/2103.01893/) (2021)
- [Embedding Compression for Teacher-to-Student Knowledge Transfer](https://mlsanity.com/paper/2402.06761/) (2024)

## Abstract (arXiv)

Recent advances in deep learning accelerated the development of content-based automatic music tagging systems. Music information retrieval (MIR) researchers proposed various architecture designs, mainly based on convolutional neural networks (CNNs), that achieve state-of-the-art results in this multi-label binary classification task. However, due to the differences in experimental setups followed by researchers, such as using different dataset splits and software versions for evaluation, it is difficult to compare the proposed architectures directly with each other. To facilitate further research, in this paper we conduct a consistent evaluation of different music tagging models on three datasets (MagnaTagATune, Million Song Dataset, and MTG-Jamendo) and provide reference results using common evaluation metrics (ROC-AUC and PR-AUC). Furthermore, all the models are evaluated with perturbed inputs to investigate the generalization capabilities concerning time stretch, pitch shift, dynamic range compression, and addition of white noise. For reproducibility, we provide the PyTorch implementations with the pre-trained models.