mlsanityData to 7 Oct 2026

Voice Filter: Few-shot text-to-speech speaker adaptation using voice conversion as a post-processing module

arXiv 2202.08164 ↗First posted 16 Feb 2022eess.AS cs.CL cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

State-of-the-art text-to-speech (TTS) systems require several hours of recorded speech data to generate high-quality synthetic speech. When using reduced amounts of training data, standard TTS models suffer from speech quality and intelligibility degradations, making training low-resource TTS systems problematic. In this paper, we propose a novel extremely low-resource TTS method called Voice Filter that uses as little as one minute of speech from a target speaker. It uses voice conversion (VC) as a post-processing module appended to a pre-existing high-quality TTS system and marks a conceptual shift in the existing TTS paradigm, framing the few-shot TTS problem as a VC task. Furthermore, we propose to use a duration-controllable TTS system to create a parallel speech corpus to facilitate the VC task. Results show that the Voice Filter outperforms state-of-the-art few-shot speech synthesis techniques in terms of objective and subjective metrics on one minute of speech on a diverse set of voices, while being competitive against a TTS model built on 30 times more data.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 16 Feb 2022.

Builds on

28 of 34 references.

References · 28

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  5. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  6. 2017Generalized End-to-End Loss for Speaker Verificationcited by 290
  7. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  8. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  9. 2018Neural Voice Cloning with a Few Samplescited by 100
  10. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  11. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 90
  12. 2018Sample Efficient Adaptive Text-to-Speechcited by 45
16 more
  1. 2020LRSpeech: Extremely Low-Resource Speech Synthesis and Recognitioncited by 43
  2. 2020VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net architecturecited by 34
  3. 2018Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesiscited by 30
  4. 2019End-to-end Text-to-speech for Low-resource Languages by Cross-Lingual Transfer Learningcited by 30
  5. 2020Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embeddingcited by 27
  6. 2020Low-resource expressive text-to-speech using data augmentationcited by 26
  7. 2020BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimizationcited by 22
  8. 2020CopyCat: Many-to-Many Fine-Grained Prosody Transfer for Neural Text-to-Speechcited by 22
  9. 2018Effect of data reduction on sequence-to-sequence neural TTScited by 16
  10. 2020Unsupervised Learning For Sequence-to-sequence Text-to-speech For Low-resource Languagescited by 16
  11. 2020From Speaker Verification to Multispeaker Speech Synthesis, Deep Transfer with Feedback Constraintcited by 14
  12. 2020AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIANcited by 14
  13. 2021Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speechcited by 14
  14. 2019High quality, lightweight and adaptable TTS using LPCNetcited by 11
  15. 2021The Multi-speaker Multi-style Voice Cloning Challenge 2021cited by 7
  16. 2021Speaker verification-derived loss and data augmentation for DNN-based multispeaker speech synthesiscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1; 2024: 2; 2025: 2.

8 papers.

  1. 2022Residual Adapters for Few-Shot Text-to-Speech Speaker Adaptationcited by 10
  2. 2022Adapter-Based Extension of Multi-Speaker Text-to-Speech Model for New Speakerscited by 9
  3. 2025Voice Cloning: Comprehensive Surveycited by 4
  4. 2022Cross-lingual Text-To-Speech with Flow-based Voice Conversion for Improved Pronunciationcited by 3
  5. 2024HyperTTS: Parameter Efficient Adaptation in Text to Speech using Hypernetworkscited by 1
  6. 2024Speech Separation with Pretrained Frontend to Minimize Domain Mismatchcited by 1
  7. 2023SCRAPS: Speech Contrastive Representations of Acoustic and Phonetic Spacescited by 0
  8. 2025Causal Self-supervised Pretrained Frontend with Predictive Code for Speech Separationcited by 0