# EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model

arXiv: https://arxiv.org/abs/2106.09317 · HTML: https://mlsanity.com/paper/2106.09317/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 37 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 25 of 25)

- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset](https://mlsanity.com/paper/2010.14794/) (2020)
- [AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines](https://mlsanity.com/paper/2010.11567/) (2020)
- [The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods](https://mlsanity.com/paper/1804.04262/) (2018)
- [The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systems](https://mlsanity.com/paper/1806.09514/) (2018)
- [LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition](https://mlsanity.com/paper/2008.03687/) (2020)
- [MultiSpeech: Multi-Speaker Text to Speech with Transformer](https://mlsanity.com/paper/2006.04664/) (2020)
- [Almost Unsupervised Text to Speech and Automatic Speech Recognition](https://mlsanity.com/paper/1905.06791/) (2019)
- [DiDiSpeech: A Large Scale Mandarin Speech Corpus](https://mlsanity.com/paper/2010.09275/) (2020)
- [Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesis](https://mlsanity.com/paper/1808.01410/) (2018)
- [DeepSinger: Singing Voice Synthesis with Data Mined From the Web](https://mlsanity.com/paper/2007.04590/) (2020)
- [Controllable Emotion Transfer For End-to-End Speech Synthesis](https://mlsanity.com/paper/2011.08679/) (2020)
- [Fine-grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesis](https://mlsanity.com/paper/2011.08477/) (2020)
- [End-to-End Emotional Speech Synthesis Using Style Tokens and Semi-Supervised Training](https://mlsanity.com/paper/1906.10859/) (2019)
- [Emotion controllable speech synthesis using emotion-unlabeled dataset with the assistance of cross-domain speech emotion recognition](https://mlsanity.com/paper/2010.13350/) (2020)
- [LSSED: a large-scale dataset and benchmark for speech emotion recognition](https://mlsanity.com/paper/2102.01754/) (2021)
- [AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIAN](https://mlsanity.com/paper/2005.05642/) (2020)
- [Using VAEs and Normalizing Flows for One-shot Text-To-Speech Synthesis of Expressive Speech](https://mlsanity.com/paper/1911.12760/) (2019)
- [The PRIORI Emotion Dataset: Linking Mood to Emotion Detected In-the-Wild](https://mlsanity.com/paper/1806.10658/) (2018)

## Built on by (most cited first, 18 of 18)

- [ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech](https://mlsanity.com/paper/2207.06389/) (2022)
- [TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation](https://mlsanity.com/paper/2205.12523/) (2022)
- [GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speech](https://mlsanity.com/paper/2205.07211/) (2022)
- [Controllable Data Generation by Deep Learning: A Review](https://mlsanity.com/paper/2207.09542/) (2022)
- [FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation](https://mlsanity.com/paper/2410.12266/) (2024)
- [VarietySound: Timbre-Controllable Video to Sound Generation via Unsupervised Information Disentanglement](https://mlsanity.com/paper/2211.10666/) (2022)
- [QI-TTS: Questioning Intonation Control for Emotional Speech Synthesis](https://mlsanity.com/paper/2303.07682/) (2023)
- [Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settings](https://mlsanity.com/paper/2407.14056/) (2024)
- [Accurate Emotion Strength Assessment for Seen and Unseen Speech Based on Data-Driven Deep Learning](https://mlsanity.com/paper/2206.07229/) (2022)
- [UMETTS: A Unified Framework for Emotional Text-to-Speech Synthesis with Multimodal Prompts](https://mlsanity.com/paper/2404.18398/) (2024)
- [Exploring speech style spaces with language models: Emotional TTS without emotion labels](https://mlsanity.com/paper/2405.11413/) (2024)
- [Emotion Selectable End-to-End Text-based Speech Editing](https://mlsanity.com/paper/2212.10191/) (2022)
- [Beyond Global Emotion: Fine-Grained Emotional Speech Synthesis with Dynamic Word-Level Modulation](https://mlsanity.com/paper/2509.20378/) (2025)
- [Cross-speaker Emotion Transfer by Manipulating Speech Style Latents](https://mlsanity.com/paper/2303.08329/) (2023)
- [Inclusivity of AI Speech in Healthcare: A Decade Look Back](https://mlsanity.com/paper/2505.10596/) (2025)
- [A Dataset for Speech Emotion Recognition in Greek Theatrical Plays](https://mlsanity.com/paper/2203.15568/) (2022)
- [Construction and Evaluation of Mandarin Multimodal Emotional Speech Database](https://mlsanity.com/paper/2401.07336/) (2024)
- [Making Social Platforms Accessible: Emotion-Aware Speech Generation with Integrated Text Analysis](https://mlsanity.com/paper/2410.19199/) (2024)

## Abstract (arXiv)

Recently, there has been an increasing interest in neural speech synthesis. While the deep neural network achieves the state-of-the-art result in text-to-speech (TTS) tasks, how to generate a more emotional and more expressive speech is becoming a new challenge to researchers due to the scarcity of high-quality emotion speech dataset and the lack of advanced emotional TTS model. In this paper, we first briefly introduce and publicly release a Mandarin emotion speech dataset including 9,724 samples with audio files and its emotion human-labeled annotation. After that, we propose a simple but efficient architecture for emotional speech synthesis called EMSpeech. Unlike those models which need additional reference audio as input, our model could predict emotion labels just from the input text and generate more expressive speech conditioned on the emotion embedding. In the experiment phase, we first validate the effectiveness of our dataset by an emotion classification task. Then we train our model on the proposed dataset and conduct a series of subjective evaluations. Finally, by showing a comparable performance in the emotional speech synthesis task, we successfully demonstrate the ability of the proposed model.