# Federated Acoustic Modeling For Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2102.04429 · HTML: https://mlsanity.com/paper/2102.04429/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2021.

## Built on by (most cited first, 20 of 20)

- [Automatic Speech Recognition using Advanced Deep Learning Approaches: A survey](https://mlsanity.com/paper/2403.01255/) (2024)
- [Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directions](https://mlsanity.com/paper/2303.02213/) (2023)
- [End-to-End Speech Recognition from Federated Acoustic Models](https://mlsanity.com/paper/2104.14297/) (2021)
- [A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness](https://mlsanity.com/paper/2212.09006/) (2022)
- [Privacy attacks for automatic speech recognition acoustic models in a federated learning framework](https://mlsanity.com/paper/2111.03777/) (2021)
- [Decoupled Federated Learning for ASR with Non-IID Data](https://mlsanity.com/paper/2206.09102/) (2022)
- [Federated Self-Supervised Learning for Acoustic Event Classification](https://mlsanity.com/paper/2203.11997/) (2022)
- [FedNST: Federated Noisy Student Training for Automatic Speech Recognition](https://mlsanity.com/paper/2206.02797/) (2022)
- [Personalized Federated Learning on Long-Tailed Data via Adversarial Feature Augmentation](https://mlsanity.com/paper/2303.15168/) (2023)
- [Private Language Model Adaptation for Speech Recognition](https://mlsanity.com/paper/2110.10026/) (2021)
- [Avoid Overfitting User Specific Information in Federated Keyword Spotting](https://mlsanity.com/paper/2206.08864/) (2022)
- [ILASR: Privacy-Preserving Incremental Learning for Automatic Speech Recognition at Production Scale](https://mlsanity.com/paper/2207.09078/) (2022)
- [Retrieving Speaker Information from Personalized Acoustic Models for Speech Recognition](https://mlsanity.com/paper/2111.04194/) (2021)
- [Adaptive Multi-Corpora Language Model Training for Speech Recognition](https://mlsanity.com/paper/2211.05121/) (2022)
- [Group Personalized Federated Learning](https://mlsanity.com/paper/2210.01863/) (2022)
- [An Experimental Study on Private Aggregation of Teacher Ensemble Learning for End-to-End Speech Recognition](https://mlsanity.com/paper/2210.05614/) (2022)
- [D4AM: A General Denoising Framework for Downstream Acoustic Models](https://mlsanity.com/paper/2311.16595/) (2023)
- [Regularized Federated Learning for Privacy-Preserving Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2506.11069/) (2025)
- [Ed-Fed: A generic federated learning framework with resource-aware client selection for edge devices](https://mlsanity.com/paper/2307.07199/) (2023)
- [Towards Personalized Federated Learning for Dysarthric Speech Recognition](https://mlsanity.com/paper/2606.13253/) (2026)

## Abstract (arXiv)

Data privacy and protection is a crucial issue for any automatic speech recognition (ASR) service provider when dealing with clients. In this paper, we investigate federated acoustic modeling using data from multiple clients. A client's data is stored on a local data server and the clients communicate only model parameters with a central server, and not their data. The communication happens infrequently to reduce the communication cost. To mitigate the non-iid issue, client adaptive federated training (CAFT) is proposed to canonicalize data across clients. The experiments are carried out on 1,150 hours of speech data from multiple domains. Hybrid LSTM acoustic models are trained via federated learning and their performance is compared to traditional centralized acoustic model training. The experimental results demonstrate the effectiveness of the proposed federated acoustic modeling strategy. We also show that CAFT can further improve the performance of the federated acoustic model.