# Towards Personalized Federated Learning for Dysarthric Speech Recognition

arXiv: https://arxiv.org/abs/2606.13253 · HTML: https://mlsanity.com/paper/2606.13253/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Builds on (most cited first, 25 of 25)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences](https://mlsanity.com/paper/1807.01647/) (2018)
- [A Survey on Distributed Machine Learning](https://mlsanity.com/paper/1912.09789/) (2019)
- [Federated Learning for Keyword Spotting](https://mlsanity.com/paper/1810.05512/) (2018)
- [Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework](https://mlsanity.com/paper/2010.15965/) (2020)
- [Training Keyword Spotting Models on Non-IID Data with Federated Learning](https://mlsanity.com/paper/2005.10406/) (2020)
- [Improving on-device speaker verification using federated learning with privacy](https://mlsanity.com/paper/2008.02651/) (2020)
- [End-to-End Speech Recognition from Federated Acoustic Models](https://mlsanity.com/paper/2104.14297/) (2021)
- [Investigation of Data Augmentation Techniques for Disordered Speech Recognition](https://mlsanity.com/paper/2201.05562/) (2022)
- [Speaker adaptation for Wav2vec2 based dysarthric ASR](https://mlsanity.com/paper/2204.00770/) (2022)
- [Federated Acoustic Modeling For Automatic Speech Recognition](https://mlsanity.com/paper/2102.04429/) (2021)
- [Speaker Adaptation Using Spectro-Temporal Deep Features for Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2202.10290/) (2022)
- [Enhancing Pre-trained ASR System Fine-tuning for Dysarthric Speech Recognition using Adversarial Data Augmentation](https://mlsanity.com/paper/2401.00662/) (2024)
- [Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2205.06445/) (2022)
- [Federated Learning in ASR: Not as Easy as You Think](https://mlsanity.com/paper/2109.15108/) (2021)
- [Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labeling](https://mlsanity.com/paper/2203.08810/) (2022)
- [Federated learning for secure development of AI models for Parkinson's disease detection using speech from different languages](https://mlsanity.com/paper/2305.11284/) (2023)
- [Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2407.13782/) (2024)
- [Importance of Smoothness Induced by Optimizers in FL4ASR: Towards Understanding Federated Learning for End-to-End ASR](https://mlsanity.com/paper/2309.13102/) (2023)
- [Communication-Efficient Personalized Federated Learning for Speech-to-Text Tasks](https://mlsanity.com/paper/2401.10070/) (2024)
- [Phone-purity Guided Discrete Tokens for Dysarthric Speech Recognition](https://mlsanity.com/paper/2501.04379/) (2025)
- [Regularized Federated Learning for Privacy-Preserving Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2506.11069/) (2025)

## Abstract (arXiv)

Speech recognition is challenging for dysarthric speakers. While federated learning (FL)-based ASR can be an effective tool for protecting privacy, it suffers from heterogeneity issues caused by speaker variability. Forcing all speakers to share the same model components can be suboptimal under such heterogeneity, making personalization a promising direction; however, related research on dysarthric speech remains limited. To this end, this paper explores two aggregation strategies to achieve personalization, including the parameter-based averaging strategy and the embedding-based averaging strategy. Experiments on UASpeech and TORGO show that the proposed methods outperform the baseline regularized FedAvg by statistically significant WER reductions of up to 0.99% absolute (3.15% relative) on UASpeech and 0.56% absolute (4.73% relative) on TORGO, respectively.