# Non-Parallel Voice Conversion for ASR Augmentation

arXiv: https://arxiv.org/abs/2209.06987 · HTML: https://mlsanity.com/paper/2209.06987/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 15 Sep 2022.

## Builds on (most cited first, 17 of 17)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss](https://mlsanity.com/paper/1905.05879/) (2019)
- [The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modeling](https://mlsanity.com/paper/2011.11588/) (2020)
- [Unsupervised Speech Decomposition via Triple Information Bottleneck](https://mlsanity.com/paper/2004.11284/) (2020)
- [Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling](https://mlsanity.com/paper/2010.04301/) (2020)
- [Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challenge](https://mlsanity.com/paper/2005.09409/) (2020)
- [Voice Conversion from Non-parallel Corpora Using Variational Auto-encoder](https://mlsanity.com/paper/1610.04019/) (2016)
- [Injecting Text in Self-Supervised Speech Pretraining](https://mlsanity.com/paper/2108.12226/) (2021)
- [Non-Parallel Sequence-to-Sequence Voice Conversion with Disentangled Linguistic and Speaker Representations](https://mlsanity.com/paper/1906.10508/) (2019)
- [Non-Parallel Voice Conversion with Cyclic Variational Autoencoder](https://mlsanity.com/paper/1907.10185/) (2019)
- [Accent and Speaker Disentanglement in Many-to-many Voice Conversion](https://mlsanity.com/paper/2011.08609/) (2020)
- [Voice Conversion Can Improve ASR in Very Low-Resource Settings](https://mlsanity.com/paper/2111.02674/) (2021)
- [Unsupervised Representation Disentanglement using Cross Domain Features and Adversarial Learning in Variational Autoencoder based Voice Conversion](https://mlsanity.com/paper/2001.07849/) (2020)
- [crank: An Open-Source Software for Nonparallel Voice Conversion Based on Vector-Quantized Variational Autoencoder](https://mlsanity.com/paper/2103.02858/) (2021)
- [Enriching Source Style Transfer in Recognition-Synthesis based Non-Parallel Voice Conversion](https://mlsanity.com/paper/2106.08741/) (2021)
- [The NeteaseGames System for Voice Conversion Challenge 2020 with Vector-quantization Variational Autoencoder and WaveNet](https://mlsanity.com/paper/2010.07630/) (2020)

## Built on by (most cited first, 2 of 2)

- [Iteratively Improving Speech Recognition and Voice Conversion](https://mlsanity.com/paper/2305.15055/) (2023)
- [Synthetic Audio Generation Framework for Air Traffic Control Speech Recognition](https://mlsanity.com/paper/2606.21340/) (2026)

## Abstract (arXiv)

Automatic speech recognition (ASR) needs to be robust to speaker differences. Voice Conversion (VC) modifies speaker characteristics of input speech. This is an attractive feature for ASR data augmentation. In this paper, we demonstrate that voice conversion can be used as a data augmentation technique to improve ASR performance, even on LibriSpeech, which contains 2,456 speakers. For ASR augmentation, it is necessary that the VC model be robust to a wide range of input speech. This motivates the use of a non-autoregressive, non-parallel VC model, and the use of a pretrained ASR encoder within the VC model. This work suggests that despite including many speakers, speaker diversity may remain a limitation to ASR quality. Finally, interrogation of our VC performance has provided useful metrics for objective evaluation of VC quality.