# Enhancing Dysarthric Speech Recognition for Unseen Speakers via Prototype-Based Adaptation

arXiv: https://arxiv.org/abs/2407.18461 · HTML: https://mlsanity.com/paper/2407.18461/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 26 Jul 2024.

## Builds on (most cited first, 15 of 15)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Personalizing ASR for Dysarthric and Accented Speech with Limited Data](https://mlsanity.com/paper/1907.13511/) (2019)
- [Why does CTC result in peaky behavior?](https://mlsanity.com/paper/2105.14849/) (2021)
- [Improved End-to-End Dysarthric Speech Recognition via Meta-learning Based Model Re-initialization](https://mlsanity.com/paper/2011.01686/) (2020)
- [Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognition](https://mlsanity.com/paper/2204.01670/) (2022)
- [kNN-CTC: Enhancing ASR via Retrieval of CTC Pseudo Labels](https://mlsanity.com/paper/2312.13560/) (2023)
- [An analysis of degenerating speech due to progressive dysarthria on ASR performance](https://mlsanity.com/paper/2211.00089/) (2022)
- [Hyper-parameter Adaptation of Conformer ASR Systems for Elderly and Dysarthric Speech Recognition](https://mlsanity.com/paper/2306.15265/) (2023)
- [Debiased Automatic Speech Recognition for Dysarthric Speech via Sample Reweighting with Sample Affinity Test](https://mlsanity.com/paper/2305.13108/) (2023)
- [Supervised Contrastive Learning with Nearest Neighbor Search for Speech Emotion Recognition](https://mlsanity.com/paper/2308.16485/) (2023)
- [MADI: Inter-domain Matching and Intra-domain Discrimination for Cross-domain Speech Recognition](https://mlsanity.com/paper/2302.11224/) (2023)

## Built on by (most cited first, 4 of 4)

- [DyPCL: Dynamic Phoneme-level Contrastive Learning for Dysarthric Speech Recognition](https://mlsanity.com/paper/2501.19010/) (2025)
- [Robust Cross-Etiology and Speaker-Independent Dysarthric Speech Recognition](https://mlsanity.com/paper/2501.14994/) (2025)
- [Bridging ASR and LLMs for Dysarthric Speech Recognition: Benchmarking Self-Supervised and Generative Approaches](https://mlsanity.com/paper/2508.08027/) (2025)
- [Analyzing Speech Condition Effects in Dysarthric ASR: A Layer-wise Probing Study](https://mlsanity.com/paper/2608.01865/) (2026)

## Abstract (arXiv)

Dysarthric speech recognition (DSR) presents a formidable challenge due to inherent inter-speaker variability, leading to severe performance degradation when applying DSR models to new dysarthric speakers. Traditional speaker adaptation methodologies typically involve fine-tuning models for each speaker, but this strategy is cost-prohibitive and inconvenient for disabled users, requiring substantial data collection. To address this issue, we introduce a prototype-based approach that markedly improves DSR performance for unseen dysarthric speakers without additional fine-tuning. Our method employs a feature extractor trained with HuBERT to produce per-word prototypes that encapsulate the characteristics of previously unseen speakers. These prototypes serve as the basis for classification. Additionally, we incorporate supervised contrastive learning to refine feature extraction. By enhancing representation quality, we further improve DSR performance, enabling effective personalized DSR. We release our code at https://github.com/NKU-HLT/PB-DSR.