# AdaVocoder: Adaptive Vocoder for Custom Voice

arXiv: https://arxiv.org/abs/2203.09825 · HTML: https://mlsanity.com/paper/2203.09825/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 37 references that are in the map.
- First posted on arXiv 18 Mar 2022.

## Builds on (most cited first, 30 of 31)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [Adversarial Audio Synthesis](https://mlsanity.com/paper/1802.04208/) (2018)
- [Efficient Neural Audio Synthesis](https://mlsanity.com/paper/1802.08435/) (2018)
- [Few-shot Image Generation via Cross-domain Correspondence](https://mlsanity.com/paper/2104.06820/) (2021)
- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [Transferring GANs: generating images from limited data](https://mlsanity.com/paper/1805.01677/) (2018)
- [AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines](https://mlsanity.com/paper/2010.11567/) (2020)
- [Neural Voice Cloning with a Few Samples](https://mlsanity.com/paper/1802.06006/) (2018)
- [AdaSpeech: Adaptive Text to Speech for Custom Voice](https://mlsanity.com/paper/2103.00993/) (2021)
- [One-Sided Unsupervised Domain Mapping](https://mlsanity.com/paper/1706.00826/) (2017)
- [Sample Efficient Adaptive Text-to-Speech](https://mlsanity.com/paper/1809.10460/) (2018)
- [Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling](https://mlsanity.com/paper/2103.14574/) (2021)
- [BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimization](https://mlsanity.com/paper/2002.01953/) (2020)
- [AdaSpeech 2: Adaptive Text to Speech with Untranscribed Data](https://mlsanity.com/paper/2104.09715/) (2021)
- [AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIAN](https://mlsanity.com/paper/2005.05642/) (2020)
- [High quality, lightweight and adaptable TTS using LPCNet](https://mlsanity.com/paper/1905.00590/) (2019)
- [AdaSpeech 3: Adaptive Text to Speech for Spontaneous Style](https://mlsanity.com/paper/2107.02530/) (2021)
- [Universal MelGAN: A Robust Neural Vocoder for High-Fidelity Waveform Generation in Multiple Domains](https://mlsanity.com/paper/2011.09631/) (2020)
- [Towards Robust Neural Vocoding for Speech Generation: A Survey](https://mlsanity.com/paper/1912.02461/) (2019)

## Built on by (most cited first, 2 of 2)

- [Voice Cloning: Comprehensive Survey](https://mlsanity.com/paper/2505.00579/) (2025)
- [Lightweight End-to-end Text-to-speech Synthesis for low resource on-device applications](https://mlsanity.com/paper/2505.07701/) (2025)

## Abstract (arXiv)

Custom voice is to construct a personal speech synthesis system by adapting the source speech synthesis model to the target model through the target few recordings. The solution to constructing a custom voice is to combine an adaptive acoustic model with a robust vocoder. However, training a robust vocoder usually requires a multi-speaker dataset, which should include various age groups and various timbres, so that the trained vocoder can be used for unseen speakers. Collecting such a multi-speaker dataset is difficult, and the dataset distribution always has a mismatch with the distribution of the target speaker dataset. This paper proposes an adaptive vocoder for custom voice from another novel perspective to solve the above problems. The adaptive vocoder mainly uses a cross-domain consistency loss to solve the overfitting problem encountered by the GAN-based neural vocoder in the transfer learning of few-shot scenes. We construct two adaptive vocoders, AdaMelGAN and AdaHiFi-GAN. First, We pre-train the source vocoder model on AISHELL3 and CSMSC datasets, respectively. Then, fine-tune it on the internal dataset VXI-children with few adaptation data. The empirical results show that a high-quality custom voice system can be built by combining a adaptive acoustic model with a adaptive vocoder.