Federated Learning in ASR: Not as Easy as You Think
Not on the map. Why?
Abstract (arXiv)
With the growing availability of smart devices and cloud services, personal speech assistance systems are increasingly used on a daily basis. Most devices redirect the voice recordings to a central server, which uses them for upgrading the recognizer model. This leads to major privacy concerns, since private data could be misused by the server or third parties. Federated learning is a decentralized optimization strategy that has been proposed to address such concerns. Utilizing this approach, private data is used for on-device training. Afterwards, updated model parameters are sent to the server to improve the global model, which is redistributed to the clients. In this work, we implement federated learning for speech recognition in a hybrid and an end-to-end model. We discuss the outcomes of these systems, which both show great similarities and only small improvements, pointing to a need for a deeper understanding of federated learning for speech recognition.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Sep 2021.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 2; 2022: 3; 2023: 3; 2024: 2; 2025: 2; 2026: 2.
14 papers.
Sort:
- 2022A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairnesscited by 13
- 2024Training Machine Learning models at the Edge: A Surveycited by 11
- 2021Privacy attacks for automatic speech recognition acoustic models in a federated learning frameworkcited by 10
- 2023Federated Learning for ASR based on Wav2vec 2.0cited by 7
- 2022Federated Domain Adaptation for ASR with Full Self-Supervisioncited by 6
- 2022Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decodingcited by 3
- 2025Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimizationcited by 3
- 2021Retrieving Speaker Information from Personalized Acoustic Models for Speech Recognitioncited by 2
- 2024Federated Learning in Healthcare: Model Misconducts, Security, Challenges, Applications, and Future Research Directions -- A Systematic Reviewcited by 1
- 2025Regularized Federated Learning for Privacy-Preserving Dysarthric and Elderly Speech Recognitioncited by 1
- 2023Ed-Fed: A generic federated learning framework with resource-aware client selection for edge devicescited by 0
- 2023Federated Representation Learning for Automatic Speech Recognitioncited by 0
- 2026SpeechLLM Meets Federated Learning for End-to-End ASR: English and Italian Case Studies28 Jul 2026
- 2026Towards Personalized Federated Learning for Dysarthric Speech Recognition11 Jun 2026
- 2025Regularized Federated Learning for Privacy-Preserving Dysarthric and Elderly Speech Recognition2 Jun 2025
- 2025Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimization11 Jan 2025
- 2024Federated Learning in Healthcare: Model Misconducts, Security, Challenges, Applications, and Future Research Directions -- A Systematic Review22 May 2024
- 2024Training Machine Learning models at the Edge: A Survey5 Mar 2024
- 2023Federated Representation Learning for Automatic Speech Recognition3 Aug 2023
- 2023Ed-Fed: A generic federated learning framework with resource-aware client selection for edge devices14 Jul 2023
- 2023Federated Learning for ASR based on Wav2vec 2.020 Feb 2023
- 2022A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness18 Dec 2022
- 2022Federated Domain Adaptation for ASR with Full Self-Supervision30 Mar 2022
- 2022Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding10 Feb 2022
- 2022A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairnesscited by 13
- 2024Training Machine Learning models at the Edge: A Surveycited by 11
- 2021Privacy attacks for automatic speech recognition acoustic models in a federated learning frameworkcited by 10
- 2023Federated Learning for ASR based on Wav2vec 2.0cited by 7
- 2022Federated Domain Adaptation for ASR with Full Self-Supervisioncited by 6
- 2022Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decodingcited by 3
- 2025Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimizationcited by 3
- 2021Retrieving Speaker Information from Personalized Acoustic Models for Speech Recognitioncited by 2
- 2024Federated Learning in Healthcare: Model Misconducts, Security, Challenges, Applications, and Future Research Directions -- A Systematic Reviewcited by 1
- 2025Regularized Federated Learning for Privacy-Preserving Dysarthric and Elderly Speech Recognitioncited by 1
- 2023Ed-Fed: A generic federated learning framework with resource-aware client selection for edge devicescited by 0
- 2023Federated Representation Learning for Automatic Speech Recognitioncited by 0
12 most cited of 14 All 14 (JSON)
- 2026SpeechLLM Meets Federated Learning for End-to-End ASR: English and Italian Case Studies28 Jul 2026
- 2026Towards Personalized Federated Learning for Dysarthric Speech Recognition11 Jun 2026
- 2025Regularized Federated Learning for Privacy-Preserving Dysarthric and Elderly Speech Recognition2 Jun 2025
- 2025Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimization11 Jan 2025
- 2024Federated Learning in Healthcare: Model Misconducts, Security, Challenges, Applications, and Future Research Directions -- A Systematic Review22 May 2024
- 2024Training Machine Learning models at the Edge: A Survey5 Mar 2024
- 2023Federated Representation Learning for Automatic Speech Recognition3 Aug 2023
- 2023Ed-Fed: A generic federated learning framework with resource-aware client selection for edge devices14 Jul 2023
- 2023Federated Learning for ASR based on Wav2vec 2.020 Feb 2023
- 2022A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness18 Dec 2022
- 2022Federated Domain Adaptation for ASR with Full Self-Supervision30 Mar 2022
- 2022Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding10 Feb 2022
12 newest of 14 All 14 (JSON)