mlsanityData to 14 Oct 2026

Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation

arXiv 2407.06310 ↗First posted 8 Jul 2024cs.SD cs.AI cs.HCcited by 2

Not on the map. Why?

Abstract (arXiv)

The application of data-intensive automatic speech recognition (ASR) technologies to dysarthric and elderly adult speech is confronted by their mismatch against healthy and nonaged voices, data scarcity and large speaker-level variability. To this end, this paper proposes two novel data-efficient methods to learn homogeneous dysarthric and elderly speaker-level features for rapid, on-the-fly test-time adaptation of DNN/TDNN and Conformer ASR models. These include: 1) speaker-level variance-regularized spectral basis embedding (VR-SBE) features that exploit a special regularization term to enforce homogeneity of speaker features in adaptation; and 2) feature-based learning hidden unit contributions (f-LHUC) transforms that are conditioned on VR-SBE features. Experiments are conducted on four tasks across two languages: the English UASpeech and TORGO dysarthric speech datasets, the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech corpora. The proposed on-the-fly speaker adaptation techniques consistently outperform baseline iVector and xVector adaptation by statistically significant word or character error rate reductions up to 5.32% absolute (18.57% relative) and batch-mode LHUC speaker adaptation by 2.24% absolute (9.20% relative), while operating with real-time factors speeding up to 33.6 times against xVectors during adaptation. The efficacy of the proposed adaptation techniques is demonstrated in a comparison against current ASR technologies including SSL pre-trained systems on UASpeech, where our best system produces a state-of-the-art WER of 23.33%. Analyses show VR-SBE features and f-LHUC transforms are insensitive to speaker-level data quantity in testtime adaptation. T-SNE visualization reveals they have stronger speaker-level homogeneity than baseline iVectors, xVectors and batch-mode LHUC transforms.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 33 of its 112 references that are in the map. First posted on arXiv 8 Jul 2024.

Builds on

33 of 112 references.

References · 33

  1. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  2. 2020Alzheimer's Dementia Recognition through Spontaneous Speech: The ADReSS Challengecited by 104
  3. 2019Personalizing ASR for Dysarthric and Accented Speech with Limited Datacited by 45
  4. 2019Personalization of End-to-end Speech Recognition On Mobile Devices For Named Entitiescited by 41
  5. 2016Learning Hidden Unit Contributions for Unsupervised Acoustic Model Adaptationcited by 40
  6. 2021On the limit of English conversational speech recognitioncited by 30
  7. 2022Investigation of Data Augmentation Techniques for Disordered Speech Recognitioncited by 27
  8. 2018Speaker-Invariant Training via Adversarial Learningcited by 23
  9. 2022Speaker adaptation for Wav2vec2 based dysarthric ASRcited by 21
  10. 2022Speaker Adaptation Using Spectro-Temporal Deep Features for Dysarthric and Elderly Speech Recognitioncited by 19
  11. 2021Adversarial Data Augmentation for Disordered Speech Recognitioncited by 18
  12. 2020Improved End-to-End Dysarthric Speech Recognition via Meta-learning Based Model Re-initializationcited by 17
21 more
  1. 2022Recent Progress in the CUHK Dysarthric Speech Recognition Systemcited by 17
  2. 2022Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognitioncited by 17
  3. 2024Enhancing Pre-trained ASR System Fine-tuning for Dysarthric Speech Recognition using Adversarial Data Augmentationcited by 17
  4. 2022Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognitioncited by 16
  5. 2022Investigating Self-supervised Pretraining Frameworks for Pathological Speech Recognitioncited by 15
  6. 2021Personalized Automatic Speech Recognition Trained on Small Disordered Speech Datasetscited by 14
  7. 2023Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognitioncited by 14
  8. 2022Spectro-Temporal Deep Features for Disordered Speech Assessment and Recognitioncited by 13
  9. 2019Speaker Adaptation for Attention-Based End-to-End Speech Recognitioncited by 12
  10. 2020Bayesian Learning for Deep Neural Network Adaptationcited by 9
  11. 2022Exploiting Cross Domain Acoustic-to-articulatory Inverted Features For Disordered Speech Recognitioncited by 9
  12. 2022Improving the Training Recipe for a Robust Conformer-based Hybrid Modelcited by 9
  13. 2023Use of Speech Impairment Severity for Dysarthric Speech Recognitioncited by 8
  14. 2020Bayesian Learning of LF-MMI Trained Time Delay Neural Networks for Speech Recognitioncited by 7
  15. 2022Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognitioncited by 7
  16. 2022Adversarial Data Augmentation Using VAE-GAN for Disordered Speech Recognitioncited by 7
  17. 2023Parameter-efficient Dysarthric Speech Recognition Using Adapter Fusion and Householder Transformationcited by 6
  18. 2023DuTa-VC: A Duration-aware Typical-to-atypical Voice Conversion Approach with Diffusion Probabilistic Modelcited by 6
  19. 2015Maximum a Posteriori Adaptation of Network Parameters in Deep Modelscited by 5
  20. 2019Speaker Adaptation for End-to-End CTC Modelscited by 5
  21. 2022On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognitioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognitioncited by 0
  2. 2026Elderly-Contextual Data Augmentation via Speech Synthesis for Elderly ASRcited by 0