# Enhancing Pre-trained ASR System Fine-tuning for Dysarthric Speech Recognition using Adversarial Data Augmentation

arXiv: https://arxiv.org/abs/2401.00662 · HTML: https://mlsanity.com/paper/2401.00662/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jan 2024.

## Built on by (most cited first, 17 of 17)

- [Enabling ASR for Low-Resource Languages: A Comprehensive Dataset Creation Approach](https://mlsanity.com/paper/2406.01446/) (2024)
- [Towards Inclusive ASR: Investigating Voice Conversion for Dysarthric Speech Recognition in Low-Resource Languages](https://mlsanity.com/paper/2505.14874/) (2025)
- [Exploring Generative Error Correction for Dysarthric Speech Recognition](https://mlsanity.com/paper/2505.20163/) (2025)
- [Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation](https://mlsanity.com/paper/2407.06310/) (2024)
- [DyPCL: Dynamic Phoneme-level Contrastive Learning for Dysarthric Speech Recognition](https://mlsanity.com/paper/2501.19010/) (2025)
- [Regularized Federated Learning for Privacy-Preserving Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2506.11069/) (2025)
- [Fairness in Dysarthric Speech Synthesis: Understanding Intrinsic Bias in Dysarthric Speech Cloning using F5-TTS](https://mlsanity.com/paper/2508.05102/) (2025)
- [StutterZero and StutterFormer: End-to-End Speech Conversion for Stuttering Transcription and Correction](https://mlsanity.com/paper/2510.18938/) (2025)
- [Adapting Foundation ASR Models to Dysarthric Speech: A Case Study](https://mlsanity.com/paper/2606.31722/) (2026)
- [On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognition](https://mlsanity.com/paper/2505.22072/) (2025)
- [Bridging ASR and LLMs for Dysarthric Speech Recognition: Benchmarking Self-Supervised and Generative Approaches](https://mlsanity.com/paper/2508.08027/) (2025)
- [Revealing the Role of Audio Channels in ASR Performance Degradation](https://mlsanity.com/paper/2508.08967/) (2025)
- [Unheard in the Digital Age: Rethinking AI Bias and Speech Diversity](https://mlsanity.com/paper/2601.18641/) (2026)
- [Towards Personalized Federated Learning for Dysarthric Speech Recognition](https://mlsanity.com/paper/2606.13253/) (2026)
- [ROMPAR: Morphological Completion and Demographic Unlearning for Romanian-Accented Speech Recognition](https://mlsanity.com/paper/2606.15984/) (2026)
- [Augmenting Dysarthric Speech Severity Assessment with MOS Supervision](https://mlsanity.com/paper/2606.18645/) (2026)
- [Improving End-to-End Speech Recognition for Dysarthric Speech through In-Domain Data Augmentation](https://mlsanity.com/paper/2606.19797/) (2026)

## Abstract (arXiv)

Automatic recognition of dysarthric speech remains a highly challenging task to date. Neuro-motor conditions and co-occurring physical disabilities create difficulty in large-scale data collection for ASR system development. Adapting SSL pre-trained ASR models to limited dysarthric speech via data-intensive parameter fine-tuning leads to poor generalization. To this end, this paper presents an extensive comparative study of various data augmentation approaches to improve the robustness of pre-trained ASR model fine-tuning to dysarthric speech. These include: a) conventional speaker-independent perturbation of impaired speech; b) speaker-dependent speed perturbation, or GAN-based adversarial perturbation of normal, control speech based on their time alignment against parallel dysarthric speech; c) novel Spectral basis GAN-based adversarial data augmentation operating on non-parallel data. Experiments conducted on the UASpeech corpus suggest GAN-based data augmentation consistently outperforms fine-tuned Wav2vec2.0 and HuBERT models using no data augmentation and speed perturbation across different data expansion operating points by statistically significant word error rate (WER) reductions up to 2.01% and 0.96% absolute (9.03% and 4.63% relative) respectively on the UASpeech test set of 16 dysarthric speakers. After cross-system outputs rescoring, the best system produced the lowest published WER of 16.53% (46.47% on very low intelligibility) on UASpeech.