mlsanityData to 7 Oct 2026

Transfer Learning Framework for Low-Resource Text-to-Speech using a Large-Scale Unlabeled Speech Corpus

arXiv 2203.15447 ↗First posted 29 Mar 2022eess.AS cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Training a text-to-speech (TTS) model requires a large scale text labeled speech corpus, which is troublesome to collect. In this paper, we propose a transfer learning framework for TTS that utilizes a large amount of unlabeled speech dataset for pre-training. By leveraging wav2vec2.0 representation, unlabeled speech can highly improve performance, especially in the lack of labeled speech. We also extend the proposed method to zero-shot multi-speaker TTS (ZS-TTS). The experimental results verify the effectiveness of the proposed method in terms of naturalness, intelligibility, and speaker generalization. We highlight that the single speaker TTS model fine-tuned on the only 10 minutes of labeled dataset outperforms the other baselines, and the ZS-TTS model fine-tuned on the only 30 minutes of single speaker dataset can generate the voice of the arbitrary speaker, by pre-training on unlabeled multi-speaker speech corpus.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 30 of its 35 references that are in the map. First posted on arXiv 29 Mar 2022.

Builds on

30 of 35 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  8. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  9. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  10. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  11. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  12. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
18 more
  1. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  2. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  3. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
  4. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  5. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  6. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  7. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  8. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 156
  9. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
  10. 2021Unsupervised Speech Recognitioncited by 147
  11. 2018Neural Voice Cloning with a Few Samplescited by 100
  12. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
  13. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  14. 2019Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddingscited by 73
  15. 2021SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Modelcited by 56
  16. 2020Clova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020cited by 36
  17. 2021Expressive Text-to-Speech using Style Tagcited by 20
  18. 2020SpeedySpeech: Efficient Neural Speech Synthesiscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 1; 2026: 1.

10 papers.

  1. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  2. 2023Transduce and Speak: Neural Transducer for Text-to-Speech with Semantic Token Predictioncited by 7
  3. 2024USAT: A Universal Speaker-Adaptive Text-to-Speech Approachcited by 6
  4. 2024Efficient Parallel Audio Generation using Group Masked Language Modelingcited by 5
  5. 2025Voice Cloning: Comprehensive Surveycited by 4
  6. 2023Pruning Self-Attention for Zero-Shot Multi-Speaker Text-to-Speechcited by 3
  7. 2023QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learningcited by 1
  8. 2024Utilizing Neural Transducers for Two-Stage Text-to-Speech via Semantic Token Predictioncited by 1
  9. 2024Maximizing Data Efficiency for Cross-Lingual TTS Adaptation by Self-Supervised Representation Mixing and Embedding Initializationcited by 0
  10. 2026Tibetan-TTS:Low-Resource Tibetan Speech Synthesis with Large Model Adaptationcited by 0