# Modeling Prosodic Phrasing with Multi-Task Learning in Tacotron-based TTS

arXiv: https://arxiv.org/abs/2008.05284 · HTML: https://mlsanity.com/paper/2008.05284/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2020.

## Built on by (most cited first, 6 of 6)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [FCTalker: Fine and Coarse Grained Context Modeling for Expressive Conversational Speech Synthesis](https://mlsanity.com/paper/2210.15360/) (2022)
- [Improving multi-speaker TTS prosody variance with a residual encoder and normalizing flows](https://mlsanity.com/paper/2106.05762/) (2021)
- [An investigation of phrase break prediction in an End-to-End TTS system](https://mlsanity.com/paper/2304.04157/) (2023)
- [A Deep Investigation of RNN and Self-attention for the Cyrillic-Traditional Mongolian Bidirectional Conversion](https://mlsanity.com/paper/2209.11963/) (2022)
- [LLM-Based Multi-Reference Evaluation for Efficient and Robust Assessment of Phrase Break Annotations](https://mlsanity.com/paper/2606.21098/) (2026)

## Abstract (arXiv)

Tacotron-based end-to-end speech synthesis has shown remarkable voice quality. However, the rendering of prosody in the synthesized speech remains to be improved, especially for long sentences, where prosodic phrasing errors can occur frequently. In this paper, we extend the Tacotron-based speech synthesis framework to explicitly model the prosodic phrase breaks. We propose a multi-task learning scheme for Tacotron training, that optimizes the system to predict both Mel spectrum and phrase breaks. To our best knowledge, this is the first implementation of multi-task learning for Tacotron based TTS with a prosodic phrasing model. Experiments show that our proposed training scheme consistently improves the voice quality for both Chinese and Mongolian systems.