# Federated Self-Supervised Learning for Acoustic Event Classification

arXiv: https://arxiv.org/abs/2203.11997 · HTML: https://mlsanity.com/paper/2203.11997/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 22 Mar 2022.

## Builds on (most cited first, 19 of 19)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Applied Federated Learning: Improving Google Keyboard Query Suggestions](https://mlsanity.com/paper/1812.02903/) (2018)
- [An Unsupervised Autoregressive Model for Speech Representation Learning](https://mlsanity.com/paper/1904.03240/) (2019)
- [Learning Problem-agnostic Speech Representations from Multiple Self-supervised Tasks](https://mlsanity.com/paper/1904.03416/) (2019)
- [Towards Learning a Universal Non-Semantic Representation of Speech](https://mlsanity.com/paper/2002.12764/) (2020)
- [Generative Pre-Training for Speech with Autoregressive Predictive Coding](https://mlsanity.com/paper/1910.12607/) (2019)
- [From Distributed Machine Learning to Federated Learning: A Survey](https://mlsanity.com/paper/2104.14362/) (2021)
- [Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection](https://mlsanity.com/paper/1702.06286/) (2017)
- [Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework](https://mlsanity.com/paper/2010.15965/) (2020)
- [Unsupervised Learning of Semantic Audio Representations](https://mlsanity.com/paper/1711.02209/) (2017)
- [Federated Self-Supervised Learning of Multi-Sensor Representations for Embedded Intelligence](https://mlsanity.com/paper/2007.13018/) (2020)
- [A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling](https://mlsanity.com/paper/1810.09050/) (2018)
- [End-to-End Speech Recognition from Federated Acoustic Models](https://mlsanity.com/paper/2104.14297/) (2021)
- [Improved Speech Representations with Multi-Target Autoregressive Predictive Coding](https://mlsanity.com/paper/2004.05274/) (2020)
- [Federated Acoustic Modeling For Automatic Speech Recognition](https://mlsanity.com/paper/2102.04429/) (2021)
- [Multi-Task Self-Supervised Pre-Training for Music Classification](https://mlsanity.com/paper/2102.03229/) (2021)
- [Self-supervised audio representation learning for mobile devices](https://mlsanity.com/paper/1905.11796/) (2019)
- [R-CRNN: Region-based Convolutional Recurrent Neural Network for Audio Event Detection](https://mlsanity.com/paper/1808.06627/) (2018)

## Built on by (most cited first, 5 of 5)

- [Mobility-Aware Federated Self-supervised Learning in Vehicular Network](https://mlsanity.com/paper/2408.00256/) (2024)
- [DRL-Based Resource Allocation for Motion Blur Resistant Federated Self-Supervised Learning in IoV](https://mlsanity.com/paper/2408.09194/) (2024)
- [Cross-device Federated Learning for Mobile Health Diagnostics: A First Study on COVID-19 Detection](https://mlsanity.com/paper/2303.07067/) (2023)
- [Exploring Federated Self-Supervised Learning for General Purpose Audio Understanding](https://mlsanity.com/paper/2402.02889/) (2024)
- [Federated Representation Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2308.02013/) (2023)

## Abstract (arXiv)

Standard acoustic event classification (AEC) solutions require large-scale collection of data from client devices for model optimization. Federated learning (FL) is a compelling framework that decouples data collection and model training to enhance customer privacy. In this work, we investigate the feasibility of applying FL to improve AEC performance while no customer data can be directly uploaded to the server. We assume no pseudo labels can be inferred from on-device user inputs, aligning with the typical use cases of AEC. We adapt self-supervised learning to the FL framework for on-device continual learning of representations, and it results in improved performance of the downstream AEC classifiers without labeled/pseudo-labeled data available. Compared to the baseline w/o FL, the proposed method improves precision up to 20.3% relatively while maintaining the recall. Our work differs from prior work in FL in that our approach does not require user-generated learning targets, and the data we use is collected from our Beta program and is de-identified, to maximally simulate the production settings.