mlsanityData to 19 Oct 2026

Differentiable and Severity-invariant Discrete Tokens for Dysarthric Speech Recognition

arXiv 2609.16855 ↗First posted 15 Sep 2026eess.AScited by 0

On the map: Speech and audio language models → ASR evaluation, bias and atypical speech

Abstract (arXiv)

This paper proposes novel differentiable and severity-invariant (DSI) discrete token approaches that are not only tightly integrated with downstream dysarthric speech recognition tasks, but also minimise discrete token diversity across speech impairment severity groups. Experiments conducted on the UASpeech and TORGO corpora suggest that Conformer models trained using the DSI tokens outperform the comparable baseline HuBERT discrete/continuous features by statistically significant WER reductions of 2.22%/0.78% absolute (9.14%/3.41% relative) and 1.78%/1.06% absolute (18.43%/11.86% relative) on the two tasks, respectively. After system combination, the lowest WERs of 18.90% and 6.38% were obtained on UASpeech and TORGO. Phoneme-specific T-SNE visualizations show that severity-invariant regularization reduces severity-dependent variation by producing greater overlap and less distinct boundaries among severity-group distributions.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in ASR evaluation, bias and atypical speech, one of 11 directions in Speech and audio language models. Builds on 32 of its 44 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

32 of 44 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Bridging ASR and LLMs for Dysarthric Speech Recognition: Benchmarking Self-Supervised and Generative Approaches11 Aug
  2. 2025On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognition28 May
  3. 2025Phone-purity Guided Discrete Tokens for Dysarthric Speech Recognition8 Jan
  4. 2024Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition3 Jul
  5. 2024Children's Speech Recognition through Discrete Token Enhancement19 Jun
  6. 2024Enhancing Pre-trained ASR System Fine-tuning for Dysarthric Speech Recognition using Adversarial Data Augmentation1 Jan
  7. 2023Use of Speech Impairment Severity for Dysarthric Speech Recognition18 May
  8. 2023Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognition28 Feb
  9. 2022Adversarial Data Augmentation Using VAE-GAN for Disordered Speech Recognition3 Nov
  10. 2022Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognition4 Apr
  11. 2022Speaker adaptation for Wav2vec2 based dysarthric ASR2 Apr
  12. 2022Speaker Adaptation Using Spectro-Temporal Deep Features for Dysarthric and Elderly Speech Recognition21 Feb
2 more
  1. 2022Investigation of Data Augmentation Techniques for Disordered Speech Recognition14 Jan
  2. 2021Adversarial Data Augmentation for Disordered Speech Recognition2 Aug

Other references · 18

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,385
  2. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,797
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,762
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 2,293
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 2,135
  6. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 754
  7. 2023Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Studycited by 61
  8. 2022VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Featurecited by 54
  9. 2023UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocodingcited by 54
  10. 2023Towards Universal Speech Discrete Tokens: A Case Study for ASR and TTScited by 44
  11. 2023DuTa-VC: A Duration-aware Typical-to-atypical Voice Conversion Approach with Diffusion Probabilistic Modelcited by 16
  12. 2019Deep clustering with concrete k-meanscited by 14
6 more
  1. 2024Estimating the Completeness of Discrete Speech Unitscited by 11
  2. 2022Two-pass Decoding and Cross-adaptation Based System Combination of End-to-end Conformer and Hybrid TDNN ASR Systemscited by 8
  3. 2025Differentiable K-means for Fully-optimized Discrete Token-based ASRcited by 6
  4. 2025Discl-VC: Disentangled Discrete Tokens and In-Context Learning for Controllable Zero-Shot Voice Conversioncited by 6
  5. 2025Accent Normalization Using Self-Supervised Discrete Tokens with Non-Parallel Datacited by 6
  6. 2024Exploring SSL Discrete Speech Features for Zipformer-based Contextual ASRcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Cross-Dataset, Age, and Gender Generalization: A Comprehensive Analysis of Fine-Tuning Strategies for Low-Resource Children's ASR
  2. 2024Speech Recognition-based Feature Extraction for Enhanced Automatic Severity Classification in Dysarthric Speech
  3. 2026Systematic Study of Dysarthric Speech Recognition: Spectral Features and Acoustic Models
  4. 2025Zero- and One-Shot Data Augmentation for Sentence-Level Dysarthric Speech Recognition in Constrained Scenarios
  5. 2026Improving End-to-End Speech Recognition for Dysarthric Speech through In-Domain Data Augmentation
  6. 2022Weak-Supervised Dysarthria-invariant Features for Spoken Language Understanding using an FHVAE and Adversarial Training
  7. 2022Automatic Severity Classification of Dysarthric speech by using Self-supervised Model with Multi-task Learning
  8. 2025Benchmarking Commercial Speech Recognition and Multimodal Large Language Models on Dysarthric Speech: Severity-Stratified Baselines and Architecture-Specific Prompting Effects