# Zero-Shot Federated Learning with New Classes for Audio Classification

arXiv: https://arxiv.org/abs/2106.10019 · HTML: https://mlsanity.com/paper/2106.10019/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 18 Jun 2021.

## Builds on (most cited first, 14 of 14)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Zero-Shot Knowledge Distillation in Deep Networks](https://mlsanity.com/paper/1905.08114/) (2019)
- [Hello Edge: Keyword Spotting on Microcontrollers](https://mlsanity.com/paper/1711.07128/) (2017)
- [Federated Learning for Keyword Spotting](https://mlsanity.com/paper/1810.05512/) (2018)
- [Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework](https://mlsanity.com/paper/2010.15965/) (2020)
- [Training Keyword Spotting Models on Non-IID Data with Federated Learning](https://mlsanity.com/paper/2005.10406/) (2020)
- [Resource-Constrained Federated Learning with Heterogeneous Labels and Models](https://mlsanity.com/paper/2011.03206/) (2020)

## Built on by (most cited first, 4 of 4)

- [Training Machine Learning models at the Edge: A Survey](https://mlsanity.com/paper/2403.02619/) (2024)
- [Federated Zero-Shot Learning for Visual Recognition](https://mlsanity.com/paper/2209.01994/) (2022)
- [Distributed Zero-Shot Learning for Visual Recognition](https://mlsanity.com/paper/2511.08170/) (2025)
- [RAST: Resolution-Aware Privileged Structure Transfer for Low-Resolution Audio Activity Recognition](https://mlsanity.com/paper/2609.38780/) (2026)

## Abstract (arXiv)

Federated learning is an effective way of extracting insights from different user devices while preserving the privacy of users. However, new classes with completely unseen data distributions can stream across any device in a federated learning setting, whose data cannot be accessed by the global server or other users. To this end, we propose a unified zero-shot framework to handle these aforementioned challenges during federated learning. We simulate two scenarios here -- 1) when the new class labels are not reported by the user, the traditional FL setting is used; 2) when new class labels are reported by the user, we synthesize Anonymized Data Impressions by calculating class similarity matrices corresponding to each device's new classes followed by unsupervised clustering to distinguish between new classes across different users. Moreover, our proposed framework can also handle statistical heterogeneities in both labels and models across the participating users. We empirically evaluate our framework on-device across different communication rounds (FL iterations) with new classes in both local and global updates, along with heterogeneous labels and models, on two widely used audio classification applications -- keyword spotting and urban sound classification, and observe an average deterministic accuracy increase of ~4.041% and ~4.258% respectively.