mlsanityData to 7 Oct 2026

Voice Conversion Can Improve ASR in Very Low-Resource Settings

arXiv 2111.02674 ↗First posted 4 Nov 2021eess.AS cs.CL cs.SDcited by 13

Not on the map. Why?

Abstract (arXiv)

Voice conversion (VC) could be used to improve speech recognition systems in low-resource languages by using it to augment limited training data. However, VC has not been widely used for this purpose because of practical issues such as compute speed and limitations when converting to and from unseen speakers. Moreover, it is still unclear whether a VC model trained on one well-resourced language can be applied to speech from another low-resource language for the aim of data augmentation. In this work we assess whether a VC system can be used cross-lingually to improve low-resource speech recognition. We combine several recent techniques to design and train a practical VC system in English, and then use this system to augment data for training speech recognition models in several low-resource languages. When using a sensible amount of VC augmented data, speech recognition performance is improved in all four low-resource languages considered. We also show that VC-based augmentation is superior to SpecAugment (a widely used signal processing augmentation method) in the low-resource languages considered.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 24 of its 39 references that are in the map. First posted on arXiv 4 Nov 2021.

Builds on

24 of 39 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  5. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  7. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  8. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  9. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  10. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  11. 2017Generalized End-to-End Loss for Speaker Verificationcited by 290
  12. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
12 more
  1. 2017Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Ratescited by 196
  2. 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 189
  3. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 185
  4. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
  5. 2020The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modelingcited by 100
  6. 2020Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversioncited by 75
  7. 2020You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentationcited by 29
  8. 2020Low-resource expressive text-to-speech using data augmentationcited by 26
  9. 2021Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech Processingcited by 26
  10. 2020Towards Natural Bilingual and Code-Switched Speech Synthesis Based on Mix of Monolingual Recordings and Cross-Lingual Voice Conversioncited by 12
  11. 2021StarGAN-ZSVC: Towards Zero-Shot Voice Conversion in Low-Resource Contextscited by 4
  12. 2019Measuring the Effectiveness of Voice Conversion on Speaker Identification and Automatic Speech Recognition Systemscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 1; 2025: 6; 2026: 2.

13 papers.

Sort:

  1. 2023Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentationcited by 30
  2. 2022ASR data augmentation in low-resource settings using cross-lingual multi-speaker TTS and cross-lingual voice conversioncited by 10
  3. 2024Improving child speech recognition with augmented child-like speechcited by 7
  4. 2025An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASRcited by 5
  5. 2025Speech Recognition for Automatically Assessing Afrikaans and isiXhosa Preschool Oral Narrativescited by 4
  6. 2022Non-Parallel Voice Conversion for ASR Augmentationcited by 2
  7. 2025Effects of Speaker Count, Duration, and Accent Diversity on Zero-Shot Accent Robustness in Low-Resource ASRcited by 2
  8. 2023Iteratively Improving Speech Recognition and Voice Conversioncited by 1
  9. 2025Improving Low-Resource Dialect Classification Using Retrieval-based Voice Conversioncited by 1
  10. 2026Gumbel-BEARD: Automatic Layer Selection for Self-Supervised Adaptation of Whisper in Low-Resource Domainscited by 1
  11. 2025From Tens of Hours to Tens of Thousands: Scaling Back-Translation for Speech Recognitioncited by 0
  12. 2025Voice Conversion Improves Cross-Domain Robustness for Spoken Arabic Dialect Identificationcited by 0
1 more
  1. 2026Synthetic Audio Generation Framework for Air Traffic Control Speech Recognitioncited by 0