# Low-resource expressive text-to-speech using data augmentation

arXiv: https://arxiv.org/abs/2011.05707 · HTML: https://mlsanity.com/paper/2011.05707/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2020.

## Built on by (most cited first, 26 of 26)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data](https://mlsanity.com/paper/2402.08093/) (2024)
- [An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era](https://mlsanity.com/paper/2210.03538/) (2022)
- [Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech](https://mlsanity.com/paper/2106.12896/) (2021)
- [Voice Conversion Can Improve ASR in Very Low-Resource Settings](https://mlsanity.com/paper/2111.02674/) (2021)
- [Cross-speaker style transfer for text-to-speech using data augmentation](https://mlsanity.com/paper/2202.05083/) (2022)
- [Voice Filter: Few-shot text-to-speech speaker adaptation using voice conversion as a post-processing module](https://mlsanity.com/paper/2202.08164/) (2022)
- [Cross-Speaker Emotion Transfer for Low-Resource Text-to-Speech Using Non-Parallel Voice Conversion with Pitch-Shift Data Augmentation](https://mlsanity.com/paper/2204.10020/) (2022)
- [Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settings](https://mlsanity.com/paper/2407.14056/) (2024)
- [Rethinking MUSHRA: Addressing Modern Challenges in Text-to-Speech Evaluation](https://mlsanity.com/paper/2411.12719/) (2024)
- [Review of end-to-end speech synthesis technology based on deep learning](https://mlsanity.com/paper/2104.09995/) (2021)
- [Distribution augmentation for low-resource expressive text-to-speech](https://mlsanity.com/paper/2202.06409/) (2022)
- [Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need](https://mlsanity.com/paper/2207.00774/) (2022)
- [Learning Emotional Representations from Imbalanced Speech Data for Speech Emotion Recognition and Emotional Text-to-Speech](https://mlsanity.com/paper/2306.05709/) (2023)
- [SingAug: Data Augmentation for Singing Voice Synthesis with Cycle-consistent Training Strategy](https://mlsanity.com/paper/2203.17001/) (2022)
- [Custom Data Augmentation for low resource ASR using Bark and Retrieval-Based Voice Conversion](https://mlsanity.com/paper/2311.14836/) (2023)
- [Automatic Evaluation of Speaker Similarity](https://mlsanity.com/paper/2207.00344/) (2022)
- [Voice Conversion for Lombard Speaking Style with Implicit and Explicit Acoustic Feature Conditioning](https://mlsanity.com/paper/2507.09310/) (2025)
- [Integrating Feedback Loss from Bi-modal Sarcasm Detector for Sarcastic Speech Synthesis](https://mlsanity.com/paper/2508.13028/) (2025)
- [Unsupervised Pre-Training For Data-Efficient Text-to-Speech On Low Resource Languages](https://mlsanity.com/paper/2303.15669/) (2023)
- [Creating New Voices using Normalizing Flows](https://mlsanity.com/paper/2312.14569/) (2023)
- [Data-augmented cross-lingual synthesis in a teacher-student framework](https://mlsanity.com/paper/2204.00061/) (2022)
- [Adapitch: Adaption Multi-Speaker Text-to-Speech Conditioned on Pitch Disentangling with Untranscribed Data](https://mlsanity.com/paper/2210.13803/) (2022)
- [Semantics-Aware Generative Latent Data Augmentation for Learning in Low-Resource Domains](https://mlsanity.com/paper/2602.02841/) (2026)
- [ZeSTA: Zero-Shot TTS Augmentation with Domain-Conditioned Training for Data-Efficient Personalized Speech Synthesis](https://mlsanity.com/paper/2603.04219/) (2026)
- [LuxEmo: Expressive Text-to-Speech Corpus for Luxembourgish](https://mlsanity.com/paper/2606.31947/) (2026)

## Abstract (arXiv)

While recent neural text-to-speech (TTS) systems perform remarkably well, they typically require a substantial amount of recordings from the target speaker reading in the desired speaking style. In this work, we present a novel 3-step methodology to circumvent the costly operation of recording large amounts of target data in order to build expressive style voices with as little as 15 minutes of such recordings. First, we augment data via voice conversion by leveraging recordings in the desired speaking style from other speakers. Next, we use that synthetic data on top of the available recordings to train a TTS model. Finally, we fine-tune that model to further increase quality. Our evaluations show that the proposed changes bring significant improvements over non-augmented models across many perceived aspects of synthesised speech. We demonstrate the proposed approach on 2 styles (newscaster and conversational), on various speakers, and on both single and multi-speaker models, illustrating the robustness of our approach.