# Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognition

arXiv: https://arxiv.org/abs/2302.14564 · HTML: https://mlsanity.com/paper/2302.14564/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 28 Feb 2023.

## Builds on (most cited first, 26 of 26)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings](https://mlsanity.com/paper/2104.03502/) (2021)
- [Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training](https://mlsanity.com/paper/2104.01027/) (2021)
- [Personalizing ASR for Dysarthric and Accented Speech with Limited Data](https://mlsanity.com/paper/1907.13511/) (2019)
- [Fine-tuning wav2vec2 for speaker recognition](https://mlsanity.com/paper/2109.15053/) (2021)
- [Investigation of Data Augmentation Techniques for Disordered Speech Recognition](https://mlsanity.com/paper/2201.05562/) (2022)
- [Speaker adaptation for Wav2vec2 based dysarthric ASR](https://mlsanity.com/paper/2204.00770/) (2022)
- [Speaker Adaptation Using Spectro-Temporal Deep Features for Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2202.10290/) (2022)
- [Improved End-to-End Dysarthric Speech Recognition via Meta-learning Based Model Re-initialization](https://mlsanity.com/paper/2011.01686/) (2020)
- [Recent Progress in the CUHK Dysarthric Speech Recognition System](https://mlsanity.com/paper/2201.05845/) (2022)
- [Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognition](https://mlsanity.com/paper/2204.01670/) (2022)
- [Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2205.06445/) (2022)
- [Investigating Self-supervised Pretraining Frameworks for Pathological Speech Recognition](https://mlsanity.com/paper/2203.15431/) (2022)
- [Personalized Automatic Speech Recognition Trained on Small Disordered Speech Datasets](https://mlsanity.com/paper/2110.04612/) (2021)
- [TaL: a synchronised multi-speaker corpus of ultrasound tongue imaging, audio, and lip videos](https://mlsanity.com/paper/2011.09804/) (2020)
- [Conformer Based Elderly Speech Recognition System for Alzheimer's Disease Detection](https://mlsanity.com/paper/2206.13232/) (2022)
- [Variational Auto-Encoder Based Variability Encoding for Dysarthric Speech Recognition](https://mlsanity.com/paper/2201.09422/) (2022)
- [Exploiting Cross Domain Acoustic-to-articulatory Inverted Features For Disordered Speech Recognition](https://mlsanity.com/paper/2203.10274/) (2022)
- [Exploring linguistic feature and model combination for speech recognition based automatic AD detection](https://mlsanity.com/paper/2206.13758/) (2022)
- [Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognition](https://mlsanity.com/paper/2206.07327/) (2022)
- [Two-pass Decoding and Cross-adaptation Based System Combination of End-to-end Conformer and Hybrid TDNN ASR Systems](https://mlsanity.com/paper/2206.11596/) (2022)

## Built on by (most cited first, 14 of 14)

- [Towards Stealthy Backdoor Attacks against Speech Recognition via Elements of Sound](https://mlsanity.com/paper/2307.08208/) (2023)
- [Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2407.13782/) (2024)
- [Use of Speech Impairment Severity for Dysarthric Speech Recognition](https://mlsanity.com/paper/2305.10659/) (2023)
- [UNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalization](https://mlsanity.com/paper/2401.14664/) (2024)
- [Exploring Pathological Speech Quality Assessment with ASR-Powered Wav2Vec2 in Data-Scarce Context](https://mlsanity.com/paper/2403.20184/) (2024)
- [Perceiver-Prompt: Flexible Speaker Adaptation in Whisper for Chinese Disordered Speech Recognition](https://mlsanity.com/paper/2406.09873/) (2024)
- [Debiased Automatic Speech Recognition for Dysarthric Speech via Sample Reweighting with Sample Affinity Test](https://mlsanity.com/paper/2305.13108/) (2023)
- [Trading Devil: Robust backdoor attack via Stochastic investment models and Bayesian approach](https://mlsanity.com/paper/2406.10719/) (2024)
- [Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation](https://mlsanity.com/paper/2407.06310/) (2024)
- [DyPCL: Dynamic Phoneme-level Contrastive Learning for Dysarthric Speech Recognition](https://mlsanity.com/paper/2501.19010/) (2025)
- [On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognition](https://mlsanity.com/paper/2505.22072/) (2025)
- [Elderly-Contextual Data Augmentation via Speech Synthesis for Elderly ASR](https://mlsanity.com/paper/2604.24770/) (2026)
- [Systematic Study of Dysarthric Speech Recognition: Spectral Features and Acoustic Models](https://mlsanity.com/paper/2606.19793/) (2026)
- [Improving End-to-End Speech Recognition for Dysarthric Speech through In-Domain Data Augmentation](https://mlsanity.com/paper/2606.19797/) (2026)

## Abstract (arXiv)

Automatic recognition of disordered and elderly speech remains a highly challenging task to date due to the difficulty in collecting such data in large quantities. This paper explores a series of approaches to integrate domain adapted SSL pre-trained models into TDNN and Conformer ASR systems for dysarthric and elderly speech recognition: a) input feature fusion between standard acoustic frontends and domain adapted wav2vec2.0 speech representations; b) frame-level joint decoding of TDNN systems separately trained using standard acoustic features alone and with additional wav2vec2.0 features; and c) multi-pass decoding involving the TDNN/Conformer system outputs to be rescored using domain adapted wav2vec2.0 models. In addition, domain adapted wav2vec2.0 representations are utilized in acoustic-to-articulatory (A2A) inversion to construct multi-modal dysarthric and elderly speech recognition systems. Experiments conducted on the UASpeech dysarthric and DementiaBank Pitt elderly speech corpora suggest TDNN and Conformer ASR systems integrated domain adapted wav2vec2.0 models consistently outperform the standalone wav2vec2.0 models by statistically significant WER reductions of 8.22% and 3.43% absolute (26.71% and 15.88% relative) on the two tasks respectively. The lowest published WERs of 22.56% (52.53% on very low intelligibility, 39.09% on unseen words) and 18.17% are obtained on the UASpeech test set of 16 dysarthric speakers, and the DementiaBank Pitt test set respectively.