# Zero-Shot Audio Classification Based on Class Label Embeddings

arXiv: https://arxiv.org/abs/1905.01926 · HTML: https://mlsanity.com/paper/1905.01926/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2019.

## Built on by (most cited first, 9 of 9)

- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [I can listen but cannot read: An evaluation of two-tower multimodal systems for instrument recognition](https://mlsanity.com/paper/2407.18058/) (2024)
- [Zero-Shot Audio Classification using Image Embeddings](https://mlsanity.com/paper/2206.04984/) (2022)
- [Improved Zero-Shot Audio Tagging & Classification with Patchout Spectrogram Transformers](https://mlsanity.com/paper/2208.11402/) (2022)
- [Multi-label Zero-Shot Audio Classification with Temporal Attention](https://mlsanity.com/paper/2409.00408/) (2024)
- [Audio Captioning with Composition of Acoustic and Semantic Information](https://mlsanity.com/paper/2105.06355/) (2021)
- [On Class Separability Pitfalls In Audio-Text Contrastive Zero-Shot Learning](https://mlsanity.com/paper/2408.13068/) (2024)
- [A Benchmark of Generative Methods for Zero-Shot Environmental Sound Classification](https://mlsanity.com/paper/2412.03771/) (2024)
- [CALM: Class-Conditional Sparse Attention Vectors for Large Audio-Language Models](https://mlsanity.com/paper/2602.07077/) (2026)

## Abstract (arXiv)

This paper proposes a zero-shot learning approach for audio classification based on the textual information about class labels without any audio samples from target classes. We propose an audio classification system built on the bilinear model, which takes audio feature embeddings and semantic class label embeddings as input, and measures the compatibility between an audio feature embedding and a class label embedding. We use VGGish to extract audio feature embeddings from audio recordings. We treat textual labels as semantic side information of audio classes, and use Word2Vec to generate class label embeddings. Results on the ESC-50 dataset show that the proposed system can perform zero-shot audio classification with small training dataset. It can achieve accuracy (26 % on average) better than random guess (10 %) on each audio category. Particularly, it reaches up to 39.7 % for the category of natural audio classes.