# Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognition

arXiv: https://arxiv.org/abs/2204.01670 · HTML: https://mlsanity.com/paper/2204.01670/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 29 references that are in the map.
- First posted on arXiv 4 Apr 2022.

## Builds on (most cited first, 10 of 10)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech](https://mlsanity.com/paper/2007.06028/) (2020)
- [Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings](https://mlsanity.com/paper/2104.03502/) (2021)
- [Exploring wav2vec 2.0 on speaker verification and language identification](https://mlsanity.com/paper/2012.06185/) (2020)
- [EasyCall corpus: a dysarthric speech dataset](https://mlsanity.com/paper/2104.02542/) (2021)
- [Simulating dysarthric speech for training data augmentation in clinical speech applications](https://mlsanity.com/paper/1804.10325/) (2018)
- [Improving Dysarthric Speech Intelligibility Using Cycle-consistent Adversarial Training](https://mlsanity.com/paper/2001.04260/) (2020)

## Built on by (most cited first, 17 of 17)

- [Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2302.14564/) (2023)
- [Wav2vec-based Detection and Severity Level Classification of Dysarthria from Speech](https://mlsanity.com/paper/2309.14107/) (2023)
- [Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2407.13782/) (2024)
- [Use of Speech Impairment Severity for Dysarthric Speech Recognition](https://mlsanity.com/paper/2305.10659/) (2023)
- [Automatic Severity Classification of Dysarthric speech by using Self-supervised Model with Multi-task Learning](https://mlsanity.com/paper/2210.15387/) (2022)
- [Enhancing Dysarthric Speech Recognition for Unseen Speakers via Prototype-Based Adaptation](https://mlsanity.com/paper/2407.18461/) (2024)
- [A Methodological and Structural Review of Parkinsons Disease Detection Across Diverse Data Modalities](https://mlsanity.com/paper/2505.00525/) (2025)
- [Debiased Automatic Speech Recognition for Dysarthric Speech via Sample Reweighting with Sample Affinity Test](https://mlsanity.com/paper/2305.13108/) (2023)
- [Investigation of Self-supervised Pre-trained Models for Classification of Voice Quality from Speech and Neck Surface Accelerometer Signals](https://mlsanity.com/paper/2308.03226/) (2023)
- [Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation](https://mlsanity.com/paper/2407.06310/) (2024)
- [Artificial intelligence-enabled detection and assessment of Parkinson's disease using multimodal data: A survey](https://mlsanity.com/paper/2502.10703/) (2025)
- [Enhancing AAC Software for Dysarthric Speakers in e-Health Settings: An Evaluation Using TORGO](https://mlsanity.com/paper/2411.00980/) (2024)
- [Robust Cross-Etiology and Speaker-Independent Dysarthric Speech Recognition](https://mlsanity.com/paper/2501.14994/) (2025)
- [Arabic Dysarthric Speech Recognition Using Adversarial and Signal-Based Augmentation](https://mlsanity.com/paper/2306.04368/) (2023)
- [Towards objective and interpretable speech disorder assessment: a comparative analysis of CNN and transformer-based models](https://mlsanity.com/paper/2406.07576/) (2024)
- [On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognition](https://mlsanity.com/paper/2505.22072/) (2025)
- [Benchmarking Human and Automatic Speech Recognition of Diverse Speech: Initial Results](https://mlsanity.com/paper/2607.19049/) (2026)

## Abstract (arXiv)

State-of-the-art automatic speech recognition (ASR) systems perform well on healthy speech. However, the performance on impaired speech still remains an issue. The current study explores the usefulness of using Wav2Vec self-supervised speech representations as features for training an ASR system for dysarthric speech. Dysarthric speech recognition is particularly difficult as several aspects of speech such as articulation, prosody and phonation can be impaired. Specifically, we train an acoustic model with features extracted from Wav2Vec, Hubert, and the cross-lingual XLSR model. Results suggest that speech representations pretrained on large unlabelled data can improve word error rate (WER) performance. In particular, features from the multilingual model led to lower WERs than filterbanks (Fbank) or models trained on a single language. Improvements were observed in English speakers with cerebral palsy caused dysarthria (UASpeech corpus), Spanish speakers with Parkinsonian dysarthria (PC-GITA corpus) and Italian speakers with paralysis-based dysarthria (EasyCall corpus). Compared to using Fbank features, XLSR-based features reduced WERs by 6.8%, 22.0%, and 7.0% for the UASpeech, PC-GITA, and EasyCall corpus, respectively.