# UNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalization

arXiv: https://arxiv.org/abs/2401.14664 · HTML: https://mlsanity.com/paper/2401.14664/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 26 Jan 2024.

## Builds on (most cited first, 15 of 15)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [Speech Resynthesis from Discrete Disentangled Self-Supervised Representations](https://mlsanity.com/paper/2104.00355/) (2021)
- [Textless Speech-to-Speech Translation on Real Data](https://mlsanity.com/paper/2112.08352/) (2021)
- [SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities](https://mlsanity.com/paper/2203.06849/) (2022)
- [Jasper: An End-to-End Convolutional Neural Acoustic Model](https://mlsanity.com/paper/1904.03288/) (2019)
- [Analysing Discrete Self Supervised Speech Representation for Spoken Language Modeling](https://mlsanity.com/paper/2301.00591/) (2023)
- [Parrotron: An End-to-End Speech-to-Speech Conversion Model and its Applications to Hearing-Impaired Speech and Speech Separation](https://mlsanity.com/paper/1904.04169/) (2019)
- [Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2302.14564/) (2023)
- [The Effectiveness of Time Stretching for Enhancing Dysarthric Speech for Improved Dysarthric Speech Recognition](https://mlsanity.com/paper/2201.04908/) (2022)
- [Improving Speech-to-Speech Translation Through Unlabeled Text](https://mlsanity.com/paper/2210.14514/) (2022)
- [Speaker Identity Preservation in Dysarthric Speech Reconstruction by Adversarial Speaker Adaptation](https://mlsanity.com/paper/2202.09082/) (2022)

## Built on by (most cited first, 8 of 8)

- [DiffDSR: Dysarthric Speech Reconstruction Using Latent Diffusion Model](https://mlsanity.com/paper/2506.00350/) (2025)
- [CoLM-DSR: Leveraging Neural Codec Language Modeling for Multi-Modal Dysarthric Speech Reconstruction](https://mlsanity.com/paper/2406.08336/) (2024)
- [Improved Intelligibility of Dysarthric Speech using Conditional Flow Matching](https://mlsanity.com/paper/2506.16127/) (2025)
- [Transcript-Free Flow-Matching Text-to-Speech via Speech Feature Conditioning](https://mlsanity.com/paper/2606.20266/) (2026)
- [AP-GRPO: Anchor-Gated Phonetic Alignment with Policy Optimization for Pathological Speech Reconstruction](https://mlsanity.com/paper/2606.15540/) (2026)
- [Augmenting Dysarthric Speech Severity Assessment with MOS Supervision](https://mlsanity.com/paper/2606.18645/) (2026)
- [Re-Sonance: A Dysarthric Asynchronous Real-Time Speech Conversion System Based on a Three-Stage Cascaded ASR-LLM-TTS Architecture](https://mlsanity.com/paper/2607.17615/) (2026)
- [Analyzing Speech Condition Effects in Dysarthric ASR: A Layer-wise Probing Study](https://mlsanity.com/paper/2608.01865/) (2026)

## Abstract (arXiv)

Dysarthric speech reconstruction (DSR) systems aim to automatically convert dysarthric speech into normal-sounding speech. The technology eases communication with speakers affected by the neuromotor disorder and enhances their social inclusion. NED-based (Neural Encoder-Decoder) systems have significantly improved the intelligibility of the reconstructed speech as compared with GAN-based (Generative Adversarial Network) approaches, but the approach is still limited by training inefficiency caused by the cascaded pipeline and auxiliary tasks of the content encoder, which may in turn affect the quality of reconstruction. Inspired by self-supervised speech representation learning and discrete speech units, we propose a Unit-DSR system, which harnesses the powerful domain-adaptation capacity of HuBERT for training efficiency improvement and utilizes speech units to constrain the dysarthric content restoration in a discrete linguistic space. Compared with NED approaches, the Unit-DSR system only consists of a speech unit normalizer and a Unit HiFi-GAN vocoder, which is considerably simpler without cascaded sub-modules or auxiliary tasks. Results on the UASpeech corpus indicate that Unit-DSR outperforms competitive baselines in terms of content restoration, reaching a 28.2% relative average word error rate reduction when compared to original dysarthric speech, and shows robustness against speed perturbation and noise.