# Federated Self-Training for Semi-Supervised Audio Recognition

arXiv: https://arxiv.org/abs/2107.06877 · HTML: https://mlsanity.com/paper/2107.06877/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 46 references that are in the map.
- First posted on arXiv 14 Jul 2021.

## Builds on (most cited first, 30 of 34)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Flower: A Friendly Federated Learning Research Framework](https://mlsanity.com/paper/2007.14390/) (2020)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization](https://mlsanity.com/paper/1807.00230/) (2018)
- [Applied Federated Learning: Improving Google Keyboard Query Suggestions](https://mlsanity.com/paper/1812.02903/) (2018)
- [Contrastive Learning of General-Purpose Audio Representations](https://mlsanity.com/paper/2010.10915/) (2020)
- [Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data](https://mlsanity.com/paper/2008.06180/) (2020)
- [Federated Learning for Emoji Prediction in a Mobile Keyboard](https://mlsanity.com/paper/1906.04329/) (2019)
- [Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning](https://mlsanity.com/paper/2006.12097/) (2020)
- [Building machines that adapt and compute like brains](https://mlsanity.com/paper/1711.04203/) (2017)
- [Federated Learning for Keyword Spotting](https://mlsanity.com/paper/1810.05512/) (2018)
- [Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Records](https://mlsanity.com/paper/1903.09296/) (2019)
- [Automatic acoustic detection of birds through deep learning: the first Bird Audio Detection challenge](https://mlsanity.com/paper/1807.05812/) (2018)
- [ToyADMOS: A Dataset of Miniature-Machine Operating Sounds for Anomalous Sound Detection](https://mlsanity.com/paper/1908.03299/) (2019)
- [Training Keyword Spotting Models on Non-IID Data with Federated Learning](https://mlsanity.com/paper/2005.10406/) (2020)
- [End-to-End Speech Recognition from Federated Acoustic Models](https://mlsanity.com/paper/2104.14297/) (2021)

## Built on by (most cited first, 5 of 5)

- [Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phases](https://mlsanity.com/paper/2203.07345/) (2022)
- [Labeling Chaos to Learning Harmony: Federated Learning with Noisy Labels](https://mlsanity.com/paper/2208.09378/) (2022)
- [Privacy-preserving Speech Emotion Recognition through Semi-Supervised Federated Learning](https://mlsanity.com/paper/2202.02611/) (2022)
- [Federated Domain Adaptation for ASR with Full Self-Supervision](https://mlsanity.com/paper/2203.15966/) (2022)
- [EmInspector: Combating Backdoor Attacks in Federated Self-Supervised Learning Through Embedding Inspection](https://mlsanity.com/paper/2405.13080/) (2024)

## Abstract (arXiv)

Federated Learning is a distributed machine learning paradigm dealing with decentralized and personal datasets. Since data reside on devices like smartphones and virtual assistants, labeling is entrusted to the clients, or labels are extracted in an automated way. Specifically, in the case of audio data, acquiring semantic annotations can be prohibitively expensive and time-consuming. As a result, an abundance of audio data remains unlabeled and unexploited on users' devices. Most existing federated learning approaches focus on supervised learning without harnessing the unlabeled data. In this work, we study the problem of semi-supervised learning of audio models via self-training in conjunction with federated learning. We propose FedSTAR to exploit large-scale on-device unlabeled data to improve the generalization of audio recognition models. We further demonstrate that self-supervised pre-trained models can accelerate the training of on-device models, significantly improving convergence to within fewer training rounds. We conduct experiments on diverse public audio classification datasets and investigate the performance of our models under varying percentages of labeled and unlabeled data. Notably, we show that with as little as 3% labeled data available, FedSTAR on average can improve the recognition rate by 13.28% compared to the fully supervised federated model.