AdaVocoder: Adaptive Vocoder for Custom Voice
Not on the map. Why?
Abstract (arXiv)
Custom voice is to construct a personal speech synthesis system by adapting the source speech synthesis model to the target model through the target few recordings. The solution to constructing a custom voice is to combine an adaptive acoustic model with a robust vocoder. However, training a robust vocoder usually requires a multi-speaker dataset, which should include various age groups and various timbres, so that the trained vocoder can be used for unseen speakers. Collecting such a multi-speaker dataset is difficult, and the dataset distribution always has a mismatch with the distribution of the target speaker dataset. This paper proposes an adaptive vocoder for custom voice from another novel perspective to solve the above problems. The adaptive vocoder mainly uses a cross-domain consistency loss to solve the overfitting problem encountered by the GAN-based neural vocoder in the transfer learning of few-shot scenes. We construct two adaptive vocoders, AdaMelGAN and AdaHiFi-GAN. First, We pre-train the source vocoder model on AISHELL3 and CSMSC datasets, respectively. Then, fine-tune it on the internal dataset VXI-children with few adaptation data. The empirical results show that a high-quality custom voice system can be built by combining a adaptive acoustic model with a adaptive vocoder.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 31 of its 37 references that are in the map. First posted on arXiv 18 Mar 2022.
Builds on
31 of 37 references.
References · 31
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2017Prototypical Networks for Few-shot Learningcited by 3,001
- 2016Matching Networks for One Shot Learningcited by 2,246
- 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
- 2018Deep Visual Domain Adaptation: A Surveycited by 634
- 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
- 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
19 more
- 2018Adversarial Audio Synthesiscited by 222
- 2018Efficient Neural Audio Synthesiscited by 218
- 2021Few-shot Image Generation via Cross-domain Correspondencecited by 206
- 2021A Survey on Neural Speech Synthesiscited by 182
- 2018Transferring GANs: generating images from limited datacited by 134
- 2020AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselinescited by 111
- 2018Neural Voice Cloning with a Few Samplescited by 100
- 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
- 2017One-Sided Unsupervised Domain Mappingcited by 91
- 2018Sample Efficient Adaptive Text-to-Speechcited by 45
- 2021Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modelingcited by 31
- 2020BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimizationcited by 22
- 2021AdaSpeech 2: Adaptive Text to Speech with Untranscribed Datacited by 17
- 2020AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIANcited by 14
- 2019High quality, lightweight and adaptable TTS using LPCNetcited by 11
- 2021AdaSpeech 3: Adaptive Text to Speech for Spontaneous Stylecited by 11
- 2020Universal MelGAN: A Robust Neural Vocoder for High-Fidelity Waveform Generation in Multiple Domainscited by 9
- 2019Towards Robust Neural Vocoding for Speech Generation: A Surveycited by 8
- 2020HooliGAN: Robust, High Quality Neural Vocodingcited by 7
Built on by
Citing papers per year: 2025: 2.
2 papers.