# Zero-shot Learning for Audio-based Music Classification and Tagging

arXiv: https://arxiv.org/abs/1907.02670 · HTML: https://mlsanity.com/paper/1907.02670/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2019.

## Built on by (most cited first, 13 of 13)

- [Contrastive Audio-Language Learning for Music](https://mlsanity.com/paper/2208.12208/) (2022)
- [Audio-visual Generalised Zero-shot Learning with Cross-modal Attention and Language](https://mlsanity.com/paper/2203.03598/) (2022)
- [MuseChat: A Conversational Music Recommendation System for Videos](https://mlsanity.com/paper/2310.06282/) (2023)
- [Language-Guided Music Recommendation for Video via Prompt Analogies](https://mlsanity.com/paper/2306.09327/) (2023)
- [Test-Time Adaptation Toward Personalized Speech Enhancement: Zero-Shot Learning with Knowledge Distillation](https://mlsanity.com/paper/2105.03544/) (2021)
- [I can listen but cannot read: An evaluation of two-tower multimodal systems for instrument recognition](https://mlsanity.com/paper/2407.18058/) (2024)
- [Improved Zero-Shot Audio Tagging & Classification with Patchout Spectrogram Transformers](https://mlsanity.com/paper/2208.11402/) (2022)
- [Multi-label Zero-Shot Audio Classification with Temporal Attention](https://mlsanity.com/paper/2409.00408/) (2024)
- [PIAST: A Multimodal Piano Dataset with Audio, Symbolic and Text](https://mlsanity.com/paper/2411.02551/) (2024)
- [MDSC: Towards Evaluating the Style Consistency Between Music and Dance](https://mlsanity.com/paper/2309.01340/) (2023)
- [On Class Separability Pitfalls In Audio-Text Contrastive Zero-Shot Learning](https://mlsanity.com/paper/2408.13068/) (2024)
- [MATT: A Multiple-instance Attention Mechanism for Long-tail Music Genre Classification](https://mlsanity.com/paper/2209.04109/) (2022)
- [Extending Music Annotation Schemas: Zero-Shot Prediction or Few-Shot Adaptation?](https://mlsanity.com/paper/2610.06920/) (2026)

## Abstract (arXiv)

Audio-based music classification and tagging is typically based on categorical supervised learning with a fixed set of labels. This intrinsically cannot handle unseen labels such as newly added music genres or semantic words that users arbitrarily choose for music retrieval. Zero-shot learning can address this problem by leveraging an additional semantic space of labels where side information about the labels is used to unveil the relationship between each other. In this work, we investigate the zero-shot learning in the music domain and organize two different setups of side information. One is using human-labeled attribute information based on Free Music Archive and OpenMIC-2018 datasets. The other is using general word semantic information based on Million Song Dataset and Last.fm tag annotations. Considering a music track is usually multi-labeled in music classification and tagging datasets, we also propose a data split scheme and associated evaluation settings for the multi-label zero-shot learning. Finally, we report experimental results and discuss the effectiveness and new possibilities of zero-shot learning in the music domain.