# Automatic Severity Classification of Dysarthric speech by using Self-supervised Model with Multi-task Learning

arXiv: https://arxiv.org/abs/2210.15387 · HTML: https://mlsanity.com/paper/2210.15387/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 29 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 9 of 9)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Detecting Dysfluencies in Stuttering Therapy Using wav2vec 2.0](https://mlsanity.com/paper/2204.03417/) (2022)
- [Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognition](https://mlsanity.com/paper/2204.01670/) (2022)
- [Investigating Self-supervised Pretraining Frameworks for Pathological Speech Recognition](https://mlsanity.com/paper/2203.15431/) (2022)
- [Spectro-Temporal Deep Features for Disordered Speech Assessment and Recognition](https://mlsanity.com/paper/2201.05554/) (2022)
- [Cross-lingual Dysarthria Severity Classification for English, Korean, and Tamil](https://mlsanity.com/paper/2209.12942/) (2022)
- [Automatic dysarthric speech detection exploiting pairwise distance-based convolutional neural networks](https://mlsanity.com/paper/2011.07545/) (2020)

## Built on by (most cited first, 6 of 6)

- [Multilingual Dysarthric Speech Assessment Using Universal Phone Recognition and Language-Specific Phonemic Contrast Modeling](https://mlsanity.com/paper/2601.21205/) (2026)
- [Speech Recognition-based Feature Extraction for Enhanced Automatic Severity Classification in Dysarthric Speech](https://mlsanity.com/paper/2412.03784/) (2024)
- [Classification of Dysarthria based on the Levels of Severity. A Systematic Review](https://mlsanity.com/paper/2310.07264/) (2023)
- [Towards objective and interpretable speech disorder assessment: a comparative analysis of CNN and transformer-based models](https://mlsanity.com/paper/2406.07576/) (2024)
- [Augmenting Dysarthric Speech Severity Assessment with MOS Supervision](https://mlsanity.com/paper/2606.18645/) (2026)
- [DSSCNet: A Transfer Learning Framework for Cross-Corpus Dysarthric Speech Severity Classification](https://mlsanity.com/paper/2606.22178/) (2026)

## Abstract (arXiv)

Automatic assessment of dysarthric speech is essential for sustained treatments and rehabilitation. However, obtaining atypical speech is challenging, often leading to data scarcity issues. To tackle the problem, we propose a novel automatic severity assessment method for dysarthric speech, using the self-supervised model in conjunction with multi-task learning. Wav2vec 2.0 XLS-R is jointly trained for two different tasks: severity classification and auxiliary automatic speech recognition (ASR). For the baseline experiments, we employ hand-crafted acoustic features and machine learning classifiers such as SVM, MLP, and XGBoost. Explored on the Korean dysarthric speech QoLT database, our model outperforms the traditional baseline methods, with a relative percentage increase of 1.25% for F1-score. In addition, the proposed model surpasses the model trained without ASR head, achieving 10.61% relative percentage improvements. Furthermore, we present how multi-task learning affects the severity classification performance by analyzing the latent representations and regularization effect.