mlsanityData to 14 Oct 2026

A Practical Recipe for Semi-Supervised Federated ASR: Online Pseudo-Labels with Server Update Stabilization

arXiv 2609.25471 ↗First posted 21 Sep 2026cs.LG cs.AI stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Semi-supervised federated learning (SSFL) trains models on clients' unlabeled data using a teacher to generate pseudo-labels, with a small labeled seed dataset on the server. Automatic Speech Recognition (ASR) is particularly fragile here: pseudo-label errors compound across the output sequence and across training rounds into divergence, leaving a large gap to fully-supervised FL. We show that closing this gap turns on two coupled design axes -- the teacher (which model generates the pseudo-labels) and the anchor (the server-side updates on labeled data that stabilize training). On the teacher axis, a per-client online teacher (each client's own evolving model) diverges on its own, but once stabilized it matches or beats the broadcast global teacher (one server model, fixed within a round) -- decisively in-domain and competitively under domain shift. As the seed grows stronger and the online teacher's advantage narrows, a transitioning teacher (global rightarrow online at round r) matches or beats both. On the anchor axis, the server must keep training on labeled data between rounds -- otherwise the online teacher drifts -- and this interleaving, more than the seed model, governs convergence. The two axes are inseparable: aggressive teacher choices pay off only once the anchor stabilizes training, which is highly sensitive to data augmentation and batch size -- the settings that govern how much input and gradient noise the server injects. How much stabilization is needed is domain-dependent, governed by the dispersion of the seed data and its overlap with client data. These findings yield guidelines for SSFL in ASR training, improving over the strongest prior method on 9 of 11 pairs, by 20.8% on average in-domain and 10.0% cross-domain, narrowing the gap to fully-supervised FL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 59 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

42 of 59 references.

References · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2016Deep Learning with Differential Privacycited by 2,728
  7. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  8. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  9. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  10. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  11. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  12. 2020Adaptive Federated Optimizationcited by 1,046
30 more
  1. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  2. 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 674
  3. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  4. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  5. 2015Listen, Attend and Spellcited by 543
  6. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 540
  7. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  8. 2022FreeMatch: Self-adaptive Thresholding for Semi-supervised Learningcited by 232
  9. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 187
  10. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 141
  11. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  12. 2020Iterative Pseudo-Labeling for Speech Recognitioncited by 83
  13. 2020SlimIPL: Language-Model-Free Iterative Pseudo-Labelingcited by 54
  14. 2020Training Speech Recognition Models with Federated Learning: A Quality/Cost Frameworkcited by 46
  15. 2021SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Trainingcited by 45
  16. 2021Momentum Pseudo-Labeling for Semi-Supervised Speech Recognitioncited by 40
  17. 2024Task Arithmetic can Mitigate Synthetic-to-Real Gap in Automatic Speech Recognitioncited by 14
  18. 2023Importance of Smoothness Induced by Optimizers in FL4ASR: Towards Understanding Federated Learning for End-to-End ASRcited by 8
  19. 2022Continuous Soft Pseudo-Labeling in ASRcited by 6
  20. 2023Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoTcited by 6
  21. 2022FedNST: Federated Noisy Student Training for Automatic Speech Recognitioncited by 5
  22. 2023Federated Self-Learning with Weak Supervision for Speech Recognitioncited by 5
  23. 2024Towards Understanding Why FixMatch Generalizes Better Than Supervised Learningcited by 5
  24. 2025An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASRcited by 5
  25. 2023When does the student surpass the teacher? Federated Semi-supervised Learning with Teacher-Student EMAcited by 4
  26. 2023Enabling Differentially Private Federated Learning for Speech Recognition: Benchmarks, Adaptive Optimizers and Gradient Clippingcited by 4
  27. 2024On the Effect of Purely Synthetic Training Data for Different Automatic Speech Recognition Architecturescited by 4
  28. 2023FedIL: Federated Incremental Learning from Decentralized Unlabeled Data with Convergence Analysiscited by 3
  29. 2024Speaker-IPL: Unsupervised Learning of Speaker Characteristics with i-Vector based Pseudo-Labelscited by 3
  30. 2026Which Data Matter? Embedding-Based Data Selection for Speech Recognitioncited by 1

Built on by

Not cited yet.