# Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis

arXiv: https://arxiv.org/abs/1808.10128 · HTML: https://mlsanity.com/paper/1808.10128/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2018.

## Built on by (most cited first, 30 of 30)

- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [Generative Pre-training for Speech with Flow Matching](https://mlsanity.com/paper/2310.16338/) (2023)
- [PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTS](https://mlsanity.com/paper/2103.15060/) (2021)
- [GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesis](https://mlsanity.com/paper/2106.15153/) (2021)
- [Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech](https://mlsanity.com/paper/2106.12896/) (2021)
- [Building African Voices](https://mlsanity.com/paper/2207.00688/) (2022)
- [Multilingual Byte2Speech Models for Scalable Low-resource Speech Synthesis](https://mlsanity.com/paper/2103.03541/) (2021)
- [KazakhTTS: An Open-Source Kazakh Text-to-Speech Synthesis Dataset](https://mlsanity.com/paper/2104.08459/) (2021)
- [Mixed-Phoneme BERT: Improving BERT with Mixed Phoneme and Sup-Phoneme Representations for Text to Speech](https://mlsanity.com/paper/2203.17190/) (2022)
- [Simple and Effective Unsupervised Speech Synthesis](https://mlsanity.com/paper/2204.02524/) (2022)
- [Voice Filter: Few-shot text-to-speech speaker adaptation using voice conversion as a post-processing module](https://mlsanity.com/paper/2202.08164/) (2022)
- [Cross-Speaker Emotion Transfer for Low-Resource Text-to-Speech Using Non-Parallel Voice Conversion with Pitch-Shift Data Augmentation](https://mlsanity.com/paper/2204.10020/) (2022)
- [When Is TTS Augmentation Through a Pivot Language Useful?](https://mlsanity.com/paper/2207.09889/) (2022)
- [Alternate Endings: Improving Prosody for Incremental Neural TTS with Predicted Future Text Input](https://mlsanity.com/paper/2102.09914/) (2021)
- [Review of end-to-end speech synthesis technology based on deep learning](https://mlsanity.com/paper/2104.09995/) (2021)
- [Distribution augmentation for low-resource expressive text-to-speech](https://mlsanity.com/paper/2202.06409/) (2022)
- [Improving Cascaded Unsupervised Speech Translation with Denoising Back-translation](https://mlsanity.com/paper/2305.07455/) (2023)
- [UniWav: Towards Unified Pre-training for Speech Representation Learning and Generation](https://mlsanity.com/paper/2503.00733/) (2025)
- [Unsupervised Data Selection for TTS: Using Arabic Broadcast News as a Case Study](https://mlsanity.com/paper/2301.09099/) (2023)
- [Learning the joint distribution of two sequences using little or no paired data](https://mlsanity.com/paper/2212.03232/) (2022)
- [A multilingual training strategy for low resource Text to Speech](https://mlsanity.com/paper/2409.01217/) (2024)
- [MnTTS: An Open-Source Mongolian Text-to-Speech Synthesis Dataset and Accompanied Baseline](https://mlsanity.com/paper/2209.10848/) (2022)
- [Semi-Supervised Learning Based on Reference Model for Low-resource TTS](https://mlsanity.com/paper/2210.14723/) (2022)
- [Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language](https://mlsanity.com/paper/2212.08321/) (2022)
- [A study on the efficacy of model pre-training in developing neural text-to-speech system](https://mlsanity.com/paper/2110.03857/) (2021)
- [AutoLV: Automatic Lecture Video Generator](https://mlsanity.com/paper/2209.08795/) (2022)
- [Unsupervised Pre-Training For Data-Efficient Text-to-Speech On Low Resource Languages](https://mlsanity.com/paper/2303.15669/) (2023)
- [QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2309.00126/) (2023)
- [Maximizing Data Efficiency for Cross-Lingual TTS Adaptation by Self-Supervised Representation Mixing and Embedding Initialization](https://mlsanity.com/paper/2402.01692/) (2024)

## Abstract (arXiv)

Although end-to-end text-to-speech (TTS) models such as Tacotron have shown excellent results, they typically require a sizable set of high-quality <text, audio> pairs for training, which are expensive to collect. In this paper, we propose a semi-supervised training framework to improve the data efficiency of Tacotron. The idea is to allow Tacotron to utilize textual and acoustic knowledge contained in large, publicly-available text and speech corpora. Importantly, these external data are unpaired and potentially noisy. Specifically, first we embed each word in the input text into word vectors and condition the Tacotron encoder on them. We then use an unpaired speech corpus to pre-train the Tacotron decoder in the acoustic domain. Finally, we fine-tune the model using available paired data. We demonstrate that the proposed framework enables Tacotron to generate intelligible speech using less than half an hour of paired training data.