mlsanityData to 14 Oct 2026

Decoupled Federated Learning for ASR with Non-IID Data

arXiv 2206.09102 ↗First posted 18 Jun 2022eess.AS cs.CL cs.DCcited by 6

Not on the map. Why?

Abstract (arXiv)

Automatic speech recognition (ASR) with federated learning (FL) makes it possible to leverage data from multiple clients without compromising privacy. The quality of FL-based ASR could be measured by recognition performance, communication and computation costs. When data among different clients are not independently and identically distributed (non-IID), the performance could degrade significantly. In this work, we tackle the non-IID issue in FL-based ASR with personalized FL, which learns personalized models for each client. Concretely, we propose two types of personalized FL approaches for ASR. Firstly, we adapt the personalization layer based FL for ASR, which keeps some layers locally to learn personalization models. Secondly, to reduce the communication and computation costs, we propose decoupled federated learning (DecoupleFL). On one hand, DecoupleFL moves the computation burden to the server, thus decreasing the computation on clients. On the other hand, DecoupleFL communicates secure high-level features instead of model parameters, thus reducing communication cost when models are large. Experiments demonstrate two proposed personalized FL-based ASR approaches could reduce WER by 2.3% - 3.4% compared with FedAvg. Among them, DecoupleFL has only 11.4% communication and 75% computation cost compared with FedAvg, which is also significantly less than the personalization layer based FL.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 30 of its 37 references that are in the map. First posted on arXiv 18 Jun 2022.

Builds on

30 of 37 references.

References · 30

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  5. 2018Federated Learning with Non-IID Datacited by 1,108
  6. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  7. 2019Federated Learning with Personalization Layerscited by 541
  8. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  9. 2020Adaptive Personalized Federated Learningcited by 358
  10. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  11. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  12. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
18 more
  1. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  2. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  3. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  4. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 141
  5. 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 108
  6. 2020Iterative Pseudo-Labeling for Speech Recognitioncited by 83
  7. 2020Training Speech Recognition Models with Federated Learning: A Quality/Cost Frameworkcited by 46
  8. 2021Momentum Pseudo-Labeling for Semi-Supervised Speech Recognitioncited by 40
  9. 2021End-to-End Speech Recognition from Federated Acoustic Modelscited by 30
  10. 2018Speaker-Invariant Training via Adversarial Learningcited by 23
  11. 2019Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?cited by 22
  12. 2021Kaizen: Continuously improving teacher using Exponential Moving Average for semi-supervised speech recognitioncited by 21
  13. 2021Federated Acoustic Modeling For Automatic Speech Recognitioncited by 20
  14. 2020Modeling Topical Relevance for Multi-Turn Dialogue Generationcited by 15
  15. 2021Partial Variable Training for Efficient On-Device Federated Learningcited by 14
  16. 2021Enabling On-Device Training of Speech Recognition Models with Federated Dropoutcited by 12
  17. 2021Attribute Inference Attack of Speech Emotion Recognition in Federated Learning Settingscited by 12
  18. 2021Privacy attacks for automatic speech recognition acoustic models in a federated learning frameworkcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 1; 2024: 2; 2025: 0; 2026: 1.

6 papers.

  1. 2024Automatic Speech Recognition using Advanced Deep Learning Approaches: A surveycited by 50
  2. 2022A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairnesscited by 13
  3. 2024Communication-Efficient Personalized Federated Learning for Speech-to-Text Taskscited by 8
  4. 2023Federated Learning for ASR based on Wav2vec 2.0cited by 7
  5. 2022End-to-end speech recognition modeling from de-identified datacited by 1
  6. 2026Federated Multilingual Speech-LLMs: Architecture and Aggregation Strategy Benchmarkingcited by 0