Unsupervised Rhythm and Voice Conversion to Improve ASR on Dysarthric Speech
Not on the map. Why?
Abstract (arXiv)
Automatic speech recognition (ASR) systems struggle with dysarthric speech due to high inter-speaker variability and slow speaking rates. To address this, we explore dysarthric-to-healthy speech conversion for improved ASR performance. Our approach extends the Rhythm and Voice (RnV) conversion framework by introducing a syllable-based rhythm modeling method suited for dysarthric speech. We assess its impact on ASR by training LF-MMI models and fine-tuning Whisper on converted speech. Experiments on the Torgo corpus reveal that LF-MMI achieves significant word error rate reductions, especially for more severe cases of dysarthria, while fine-tuning Whisper on converted data has minimal effect on its performance. These results highlight the potential of unsupervised rhythm and voice conversion for dysarthric ASR. Code available at: https://github.com/idiap/RnV
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 8 of its 20 references that are in the map. First posted on arXiv 2 Jun 2025.
Builds on
8 of 20 references.
References · 8
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2023Voice Conversion With Just Nearest Neighborscited by 68
- 2024Training Data Augmentation for Dysarthric Automatic Speech Recognition by Text-to-Dysarthric-Speech Synthesiscited by 14
- 2022Synthesizing Dysarthric Speech Using Multi-talker TTS for Dysarthric Speech Recognitioncited by 10
- 2023Rhythm Modeling for Voice Conversioncited by 6
- 2025Unsupervised Rhythm and Voice Conversion of Dysarthric to Healthy Speech for ASRcited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.