# A Benchmark of Generative Methods for Zero-Shot Environmental Sound Classification

arXiv: https://arxiv.org/abs/2412.03771 · HTML: https://mlsanity.com/paper/2412.03771/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 26 of its 52 references that are in the map.
- First posted on arXiv 4 Dec 2024.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [Your Diffusion Model is Secretly a Zero-Shot Classifier](https://mlsanity.com/paper/2303.16203/) (2023)
- [Feature Generating Networks for Zero-Shot Learning](https://mlsanity.com/paper/1712.00981/) (2017)
- [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017)
- [Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders](https://mlsanity.com/paper/1812.01784/) (2018)
- [Label-Embedding for Image Classification](https://mlsanity.com/paper/1503.08677/) (2015)
- [Text-to-Image Diffusion Models are Zero-Shot Classifiers](https://mlsanity.com/paper/2303.15233/) (2023)
- [A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts](https://mlsanity.com/paper/1712.01381/) (2017)
- [Leveraging the Invariant Side of Generative Zero-Shot Learning](https://mlsanity.com/paper/1904.04092/) (2019)
- [A multi-device dataset for urban acoustic scene classification](https://mlsanity.com/paper/1807.09840/) (2018)
- [A Generative Model For Zero Shot Learning Using Conditional Variational Autoencoders](https://mlsanity.com/paper/1709.00663/) (2017)
- [Generative Dual Adversarial Network for Generalized Zero-shot Learning](https://mlsanity.com/paper/1811.04857/) (2018)
- [A Closer Look at Self-training for Zero-Label Semantic Segmentation](https://mlsanity.com/paper/2104.11692/) (2021)
- [Learning Robust Visual-Semantic Embeddings](https://mlsanity.com/paper/1703.05908/) (2017)
- [Zero-Shot Instance Segmentation](https://mlsanity.com/paper/2104.06601/) (2021)
- [Zero-Shot Audio Classification via Semantic Embeddings](https://mlsanity.com/paper/2011.12133/) (2020)
- [Task Aligned Generative Meta-learning for Zero-shot Learning](https://mlsanity.com/paper/2103.02185/) (2021)
- [ZeroDiff: Solidified Visual-Semantic Correlation in Zero-Shot Learning](https://mlsanity.com/paper/2406.02929/) (2024)
- [Zero-Shot Audio Classification Based on Class Label Embeddings](https://mlsanity.com/paper/1905.01926/) (2019)
- [Zero-Shot Audio Classification with Factored Linear and Nonlinear Acoustic-Semantic Projections](https://mlsanity.com/paper/2011.12657/) (2020)
- [C-SAW: Self-Supervised Prompt Learning for Image Generalization in Remote Sensing](https://mlsanity.com/paper/2311.15812/) (2023)
- [Deep Matching Autoencoders](https://mlsanity.com/paper/1711.06047/) (2017)
- [ARCA23K: An audio dataset for investigating open-set label noise](https://mlsanity.com/paper/2109.09227/) (2021)

## Abstract (arXiv)

Zero-shot learning enables models to generalise to unseen classes using semantic information, bridging the gap between training classes and previously unseen test classes. While widely studied in computer vision, its application to environmental audio remains underexplored, and generative approaches have received little attention. This work presents the first benchmark of generative methods for zero-shot environmental sound classification. Four approaches spanning variational, adversarial, diffusion-based, and denoising paradigms are evaluated. The benchmark includes CADA-VAE and LisGAN, adapted from computer vision, together with two embedding-generation methods introduced in this work: one based on a denoising diffusion probabilistic model (DDPM) and the other on a conditional generative denoising network (CGDN). Experiments on five environmental audio datasets (ESC-50, ARCA23K-FSD, FSC22, UrbanSound8K, and TAU Urban Acoustic Scenes 2019) and one music dataset (GTZAN) show that generative methods are competitive with established compatibility-based approaches. Among the evaluated generative methods, CGDN achieves the highest average accuracy and is the only one to significantly outperform both the DDPM- and GAN-based methods, while remaining statistically indistinguishable from the strong ALE baseline. These findings suggest that optimisation stability is an important factor in generative zero-shot learning for environmental audio.