# Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability

arXiv: https://arxiv.org/abs/2104.01408 · HTML: https://mlsanity.com/paper/2104.01408/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 3 Apr 2021.

## Builds on (most cited first, 21 of 21)

- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron](https://mlsanity.com/paper/1803.09047/) (2018)
- [Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset](https://mlsanity.com/paper/2010.14794/) (2020)
- [An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning](https://mlsanity.com/paper/2008.03648/) (2020)
- [Hierarchical Generative Modeling for Controllable Speech Synthesis](https://mlsanity.com/paper/1810.07217/) (2018)
- [A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer](https://mlsanity.com/paper/1905.10060/) (2019)
- [Learning latent representations for style control and transfer in end-to-end speech synthesis](https://mlsanity.com/paper/1812.04342/) (2018)
- [Controllable Emotion Transfer For End-to-End Speech Synthesis](https://mlsanity.com/paper/2011.08679/) (2020)
- [Expressive TTS Training with Frame and Style Reconstruction Loss](https://mlsanity.com/paper/2008.01490/) (2020)
- [Emotional speech synthesis with rich and granularized control](https://mlsanity.com/paper/1911.01635/) (2019)
- [End-to-End Emotional Speech Synthesis Using Style Tokens and Semi-Supervised Training](https://mlsanity.com/paper/1906.10859/) (2019)
- [Emotion controllable speech synthesis using emotion-unlabeled dataset with the assistance of cross-domain speech emotion recognition](https://mlsanity.com/paper/2010.13350/) (2020)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Teacher-Student Training for Robust Tacotron-based TTS](https://mlsanity.com/paper/1911.02839/) (2019)
- [Reinforcement Learning Based Speech Enhancement for Robust Speech Recognition](https://mlsanity.com/paper/1811.04224/) (2018)
- [Reinforcement Learning of Speech Recognition System Based on Policy Gradient and Hypothesis Selection](https://mlsanity.com/paper/1711.03689/) (2017)
- [DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score](https://mlsanity.com/paper/1810.09137/) (2018)
- [A Machine of Few Words -- Interactive Speaker Recognition with Reinforcement Learning](https://mlsanity.com/paper/2008.03127/) (2020)

## Built on by (most cited first, 19 of 19)

- [Emotional Voice Conversion: Theory, Databases and ESD](https://mlsanity.com/paper/2105.14762/) (2021)
- [SpeechAlign: Aligning Speech Generation to Human Preferences](https://mlsanity.com/paper/2404.05600/) (2024)
- [Emotion Intensity and its Control for Emotional Voice Conversion](https://mlsanity.com/paper/2201.03967/) (2022)
- [An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era](https://mlsanity.com/paper/2210.03538/) (2022)
- [Controllable Data Generation by Deep Learning: A Review](https://mlsanity.com/paper/2207.09542/) (2022)
- [Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalities](https://mlsanity.com/paper/2210.15359/) (2022)
- [Fine-grained style control in Transformer-based Text-to-speech Synthesis](https://mlsanity.com/paper/2110.06306/) (2021)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlook](https://mlsanity.com/paper/2210.13623/) (2022)
- [Explore 3D Dance Generation via Reward Model from Automatically-Ranked Demonstrations](https://mlsanity.com/paper/2312.11442/) (2023)
- [YingMusic-Singer: Zero-shot Singing Voice Synthesis and Editing with Annotation-free Melody Guidance](https://mlsanity.com/paper/2512.04779/) (2025)
- [Exploring speech style spaces with language models: Emotional TTS without emotion labels](https://mlsanity.com/paper/2405.11413/) (2024)
- [StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis](https://mlsanity.com/paper/2110.03156/) (2021)
- [HPRO: Hierarchical Progressive Reward Optimization via Preference Extraction for Emotional Text-to-Speech](https://mlsanity.com/paper/2606.28249/) (2026)
- [Computational Charisma -- A Brick by Brick Blueprint for Building Charismatic Artificial Intelligence](https://mlsanity.com/paper/2301.00142/) (2022)
- [MnTTS2: An Open-Source Multi-Speaker Mongolian Text-to-Speech Synthesis Dataset](https://mlsanity.com/paper/2301.00657/) (2022)
- [Rhythm-controllable Attention with High Robustness for Long Sentence Speech Synthesis](https://mlsanity.com/paper/2306.02593/) (2023)
- [HuLA: Prosody-Aware Anti-Spoofing with Multi-Task Learning for Expressive and Emotional Synthetic Speech](https://mlsanity.com/paper/2509.21676/) (2025)
- [TokAN: Accent Normalization Using Self-Supervised Speech Tokens](https://mlsanity.com/paper/2607.03928/) (2026)

## Abstract (arXiv)

Emotional text-to-speech synthesis (ETTS) has seen much progress in recent years. However, the generated voice is often not perceptually identifiable by its intended emotion category. To address this problem, we propose a new interactive training paradigm for ETTS, denoted as i-ETTS, which seeks to directly improve the emotion discriminability by interacting with a speech emotion recognition (SER) model. Moreover, we formulate an iterative training strategy with reinforcement learning to ensure the quality of i-ETTS optimization. Experimental results demonstrate that the proposed i-ETTS outperforms the state-of-the-art baselines by rendering speech with more accurate emotion style. To our best knowledge, this is the first study of reinforcement learning in emotional text-to-speech synthesis.