# A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion

arXiv: https://arxiv.org/abs/2106.01415 · HTML: https://mlsanity.com/paper/2106.01415/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [Voice Conversion from Non-parallel Corpora Using Variational Auto-encoder](https://mlsanity.com/paper/1610.04019/) (2016)
- [Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks](https://mlsanity.com/paper/1704.00849/) (2017)
- [Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretraining](https://mlsanity.com/paper/1912.06813/) (2019)
- [Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations](https://mlsanity.com/paper/1804.02812/) (2018)
- [AttS2S-VC: Sequence-to-Sequence Voice Conversion with Attention and Context Preservation Mechanisms](https://mlsanity.com/paper/1811.04076/) (2018)
- [Parrotron: An End-to-End Speech-to-Speech Conversion Model and its Applications to Hearing-Impaired Speech and Speech Separation](https://mlsanity.com/paper/1904.04169/) (2019)
- [Sequence-to-Sequence Acoustic Modeling for Voice Conversion](https://mlsanity.com/paper/1810.06865/) (2018)
- [Non-Parallel Voice Conversion with Cyclic Variational Autoencoder](https://mlsanity.com/paper/1907.10185/) (2019)
- [The 2020 ESPnet update: new features, broadened applications, performance improvements, and future plans](https://mlsanity.com/paper/2012.13006/) (2020)
- [Unsupervised Representation Disentanglement using Cross Domain Features and Adversarial Learning in Variational Autoencoder based Voice Conversion](https://mlsanity.com/paper/2001.07849/) (2020)
- [Pretraining Techniques for Sequence-to-Sequence Voice Conversion](https://mlsanity.com/paper/2008.03088/) (2020)
- [Improved Lite Audio-Visual Speech Enhancement](https://mlsanity.com/paper/2008.13222/) (2020)
- [Improving Sequence-to-Sequence Acoustic Modeling by Adding Text-Supervision](https://mlsanity.com/paper/1811.08111/) (2018)
- [Many-to-Many Voice Transformer Network](https://mlsanity.com/paper/2005.08445/) (2020)
- [crank: An Open-Source Software for Nonparallel Voice Conversion Based on Vector-Quantized Variational Autoencoder](https://mlsanity.com/paper/2103.02858/) (2021)

## Built on by (most cited first, 6 of 6)

- [Towards Identity Preserving Normal to Dysarthric Voice Conversion](https://mlsanity.com/paper/2110.08213/) (2021)
- [Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2407.13782/) (2024)
- [The Effectiveness of Time Stretching for Enhancing Dysarthric Speech for Improved Dysarthric Speech Recognition](https://mlsanity.com/paper/2201.04908/) (2022)
- [Speaker Identity Preservation in Dysarthric Speech Reconstruction by Adversarial Speaker Adaptation](https://mlsanity.com/paper/2202.09082/) (2022)
- [Evaluating Methods for Ground-Truth-Free Foreign Accent Conversion](https://mlsanity.com/paper/2309.02133/) (2023)
- [ClaritySpeech: Dementia Obfuscation in Speech](https://mlsanity.com/paper/2507.09282/) (2025)

## Abstract (arXiv)

We propose a new paradigm for maintaining speaker identity in dysarthric voice conversion (DVC). The poor quality of dysarthric speech can be greatly improved by statistical VC, but as the normal speech utterances of a dysarthria patient are nearly impossible to collect, previous work failed to recover the individuality of the patient. In light of this, we suggest a novel, two-stage approach for DVC, which is highly flexible in that no normal speech of the patient is required. First, a powerful parallel sequence-to-sequence model converts the input dysarthric speech into a normal speech of a reference speaker as an intermediate product, and a nonparallel, frame-wise VC model realized with a variational autoencoder then converts the speaker identity of the reference speech back to that of the patient while assumed to be capable of preserving the enhanced quality. We investigate several design options. Experimental evaluation results demonstrate the potential of our approach to improving the quality of the dysarthric speech while maintaining the speaker identity.