# Music Genre Classification using Large Language Models

arXiv: https://arxiv.org/abs/2410.08321 · HTML: https://mlsanity.com/paper/2410.08321/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Builds on (most cited first, 16 of 16)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [Comparative layer-wise analysis of self-supervised speech models](https://mlsanity.com/paper/2211.03929/) (2022)
- [Convolutional Recurrent Neural Networks for Music Classification](https://mlsanity.com/paper/1609.04243/) (2016)
- [MARBLE: Music Audio Representation Benchmark for Universal Evaluation](https://mlsanity.com/paper/2306.10548/) (2023)
- [A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognition](https://mlsanity.com/paper/2203.14779/) (2022)
- [End-to-end learning for music audio tagging at scale](https://mlsanity.com/paper/1711.02520/) (2017)
- [End-to-End Environmental Sound Classification using a 1D Convolutional Neural Network](https://mlsanity.com/paper/1904.08990/) (2019)
- [Deep convolutional neural networks for predominant instrument recognition in polyphonic music](https://mlsanity.com/paper/1605.09507/) (2016)
- [1D CNN Architectures for Music Genre Classification](https://mlsanity.com/paper/2105.07302/) (2021)
- [A Survey of Foundation Models for Music Understanding](https://mlsanity.com/paper/2409.09601/) (2024)
- [Evaluation of pretrained language models on music understanding](https://mlsanity.com/paper/2409.11449/) (2024)

## Abstract (arXiv)

This paper exploits the zero-shot capabilities of pre-trained large language models (LLMs) for music genre classification. The proposed approach splits audio signals into 20 ms chunks and processes them through convolutional feature encoders, a transformer encoder, and additional layers for coding audio units and generating feature vectors. The extracted feature vectors are used to train a classification head. During inference, predictions on individual chunks are aggregated for a final genre classification. We conducted a comprehensive comparison of LLMs, including WavLM, HuBERT, and wav2vec 2.0, with traditional deep learning architectures like 1D and 2D convolutional neural networks (CNNs) and the audio spectrogram transformer (AST). Our findings demonstrate the superior performance of the AST model, achieving an overall accuracy of 85.5%, surpassing all other models evaluated. These results highlight the potential of LLMs and transformer-based architectures for advancing music information retrieval tasks, even in zero-shot scenarios.