mlsanityData to 14 Oct 2026

DiffDSR: Dysarthric Speech Reconstruction Using Latent Diffusion Model

arXiv 2506.00350 ↗First posted 31 May 2025cs.SD eess.AScited by 5

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Dysarthric speech reconstruction (DSR) aims to convert dysarthric speech into comprehensible speech while maintaining the speaker's identity. Despite significant advancements, existing methods often struggle with low speech intelligibility and poor speaker similarity. In this study, we introduce a novel diffusion-based DSR system that leverages a latent diffusion model to enhance the quality of speech reconstruction. Our model comprises: (i) a speech content encoder for phoneme embedding restoration via pre-trained self-supervised learning (SSL) speech foundation models; (ii) a speaker identity encoder for speaker-aware identity preservation by in-context learning mechanism; (iii) a diffusion-based speech generator to reconstruct the speech based on the restored phoneme embedding and preserved speaker identity. Through evaluations on the widely-used UASpeech corpus, our proposed model shows notable enhancements in speech intelligibility and speaker similarity.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 23 of its 35 references that are in the map. First posted on arXiv 31 May 2025.

Builds on

23 of 35 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025ZSVC: Zero-shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial Training8 Jan
  2. 2024SimpleSpeech: Towards Simple and Efficient Text-to-Speech with Scalar Latent Transformer Diffusion Models4 Jun
  3. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models5 Mar
  4. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers18 Apr

Other references · 19

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  2. 2022Scalable Diffusion Models with Transformerscited by 6,054
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  5. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  6. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  8. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  9. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  10. 2022High Fidelity Neural Audio Compressioncited by 687
  11. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  12. 2017Generalized End-to-End Loss for Speaker Verificationcited by 293
7 more
  1. 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 122
  2. 2024UNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalizationcited by 8
  3. 2020Learning Explicit Prosody Models and Deep Speaker Embeddings for Atypical Voice Conversioncited by 5
  4. 2024Target Speech Extraction with Pre-trained AV-HuBERT and Mask-And-Recover Strategycited by 5
  5. 2023StyleSpeech: Self-supervised Style Enhancing with VQ-VAE-based Pre-training for Expressive Audiobook Speech Synthesiscited by 4
  6. 2024CoLM-DSR: Leveraging Neural Codec Language Modeling for Multi-Modal Dysarthric Speech Reconstructioncited by 4
  7. 2024Exploiting Audio-Visual Features with Pretrained AV-HuBERT for Multi-Modal Dysarthric Speech Reconstructioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Transcript-Free Flow-Matching Text-to-Speech via Speech Feature Conditioningcited by 1
  2. 2026An Evaluation Framework for Text-to-Speech Voice Reconstructioncited by 1
  3. 2025ELEGANCE: Efficient LLM Guidance for Audio-Visual Target Speech Extractioncited by 0
  4. 2026AP-GRPO: Anchor-Gated Phonetic Alignment with Policy Optimization for Pathological Speech Reconstructioncited by 0
  5. 2026Analyzing Speech Condition Effects in Dysarthric ASR: A Layer-wise Probing Studycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Objective and Subjective Evaluation of Diffusion-Based Speech Enhancement for Dysarthric Speech
  2. 2024Training Data Augmentation for Dysarthric Automatic Speech Recognition by Text-to-Dysarthric-Speech Synthesis
  3. 2026End-to-End Simultaneous Dysarthric Speech Reconstruction with Frame-Level Adaptor and Multiple Wait-k Knowledge Distillation
  4. 2025Improved Intelligibility of Dysarthric Speech using Conditional Flow Matching
  5. 2023DuTa-VC: A Duration-aware Typical-to-atypical Voice Conversion Approach with Diffusion Probabilistic Model
  6. 2026DARS: Dysarthria-Aware Rhythm-Style Synthesis for ASR Enhancement
  7. 2025Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens
  8. 2023Creating Personalized Synthetic Voices from Post-Glossectomy Speech with Guided Diffusion Models