# End-to-End Emotional Speech Synthesis Using Style Tokens and Semi-Supervised Training

arXiv: https://arxiv.org/abs/1906.10859 · HTML: https://mlsanity.com/paper/1906.10859/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2019.

## Built on by (most cited first, 19 of 19)

- [Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speech](https://mlsanity.com/paper/2103.04088/) (2021)
- [An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era](https://mlsanity.com/paper/2210.03538/) (2022)
- [Speech Synthesis with Mixed Emotions](https://mlsanity.com/paper/2208.05890/) (2022)
- [Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability](https://mlsanity.com/paper/2104.01408/) (2021)
- [Towards Multi-Scale Style Control for Expressive Speech Synthesis](https://mlsanity.com/paper/2104.03521/) (2021)
- [EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model](https://mlsanity.com/paper/2106.09317/) (2021)
- [Self-supervised Context-aware Style Representation for Expressive Speech Synthesis](https://mlsanity.com/paper/2206.12559/) (2022)
- [A Review of Human Emotion Synthesis Based on Generative Technology](https://mlsanity.com/paper/2412.07116/) (2024)
- [Exploring speech style spaces with language models: Emotional TTS without emotion labels](https://mlsanity.com/paper/2405.11413/) (2024)
- [Marco-Voice Technical Report](https://mlsanity.com/paper/2508.02038/) (2025)
- [Semi-supervised learning for continuous emotional intensity controllable speech synthesis with disentangled representations](https://mlsanity.com/paper/2211.06160/) (2022)
- [Cross-speaker Emotion Transfer by Manipulating Speech Style Latents](https://mlsanity.com/paper/2303.08329/) (2023)
- [Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Models](https://mlsanity.com/paper/2603.17231/) (2026)
- [UniTTS: Residual Learning of Unified Embedding Space for Speech Style Control](https://mlsanity.com/paper/2106.11171/) (2021)
- [Multilingual Emotion Neurons in Large Audio-Language Models](https://mlsanity.com/paper/2608.08772/) (2026)
- [Code-Mixed Text to Speech Synthesis under Low-Resource Constraints](https://mlsanity.com/paper/2312.01103/) (2023)
- [AMNet: An Acoustic Model Network for Enhanced Mandarin Speech Synthesis](https://mlsanity.com/paper/2504.09225/) (2025)
- [Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Models](https://mlsanity.com/paper/2608.17102/) (2026)
- [Alignment-Free Text-Audiobox for Voice Dubbing and Full-Duplex Dialogue Synthesis](https://mlsanity.com/paper/2609.03992/) (2026)

## Abstract (arXiv)

This paper proposes an end-to-end emotional speech synthesis (ESS) method which adopts global style tokens (GSTs) for semi-supervised training. This model is built based on the GST-Tacotron framework. The style tokens are defined to present emotion categories. A cross entropy loss function between token weights and emotion labels is designed to obtain the interpretability of style tokens utilizing the small portion of training data with emotion labels. Emotion recognition experiments confirm that this method can achieve one-to-one correspondence between style tokens and emotion categories effectively. Objective and subjective evaluation results show that our model outperforms the conventional Tacotron model for ESS when only 5% of training data has emotion labels. Its subjective performance is close to the Tacotron model trained using all emotion labels.