# EmoSpeech: A Corpus of Emotionally Rich and Contextually Detailed Speech Annotations

arXiv: https://arxiv.org/abs/2412.06581 · HTML: https://mlsanity.com/paper/2412.06581/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 9 Dec 2024.

## Builds on (most cited first, 19 of 19)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers](https://mlsanity.com/paper/2304.09116/) (2023)
- [Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset](https://mlsanity.com/paper/2010.14794/) (2020)
- [PromptTTS: Controllable Text-to-Speech with Text Descriptions](https://mlsanity.com/paper/2211.12171/) (2022)
- [FunASR: A Fundamental End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2305.11013/) (2023)
- [InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Prompt](https://mlsanity.com/paper/2301.13662/) (2023)
- [CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model](https://mlsanity.com/paper/2305.06908/) (2023)
- [EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance](https://mlsanity.com/paper/2211.09496/) (2022)
- [SECap: Speech Emotion Captioning with Large Language Model](https://mlsanity.com/paper/2312.10381/) (2023)
- [BridgeNet: A Joint Learning Network of Depth Map Super-Resolution and Monocular Depth Estimation](https://mlsanity.com/paper/2107.12541/) (2021)
- [Emotional speech synthesis with rich and granularized control](https://mlsanity.com/paper/1911.01635/) (2019)
- [FlashSpeech: Efficient Zero-Shot Speech Synthesis](https://mlsanity.com/paper/2404.14700/) (2024)
- [Multi-Modal Emotion recognition on IEMOCAP Dataset using Deep Learning](https://mlsanity.com/paper/1804.05788/) (2018)
- [LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning](https://mlsanity.com/paper/2305.18169/) (2023)
- [LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioning](https://mlsanity.com/paper/2406.07969/) (2024)
- [Language Model-Based Emotion Prediction Methods for Emotional Speech Synthesis Systems](https://mlsanity.com/paper/2206.15067/) (2022)
- [An overview of text-to-speech systems and media applications](https://mlsanity.com/paper/2310.14301/) (2023)

## Built on by (most cited first, 1 of 1)

- [COT-TTS: Audio Context-Aware Text-to-Speech with Chain-of-Thought Reasoning](https://mlsanity.com/paper/2609.22697/) (2026)

## Abstract (arXiv)

Advances in text-to-speech (TTS) technology have significantly improved the quality of generated speech, closely matching the timbre and intonation of the target speaker. However, due to the inherent complexity of human emotional expression, the development of TTS systems capable of controlling subtle emotional differences remains a formidable challenge. Existing emotional speech databases often suffer from overly simplistic labelling schemes that fail to capture a wide range of emotional states, thus limiting the effectiveness of emotion synthesis in TTS applications. To this end, recent efforts have focussed on building databases that use natural language annotations to describe speech emotions. However, these approaches are costly and require more emotional depth to train robust systems. In this paper, we propose a novel process aimed at building databases by systematically extracting emotion-rich speech segments and annotating them with detailed natural language descriptions through a generative model. This approach enhances the emotional granularity of the database and significantly reduces the reliance on costly manual annotations by automatically augmenting the data with high-level language models. The resulting rich database provides a scalable and economically viable solution for developing a more nuanced and dynamic basis for developing emotionally controlled TTS systems.