# Generative Emotional AI for Speech Emotion Recognition: The Case for Synthetic Emotional Speech Augmentation

arXiv: https://arxiv.org/abs/2301.03751 · HTML: https://mlsanity.com/paper/2301.03751/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jan 2023.

## Built on by (most cited first, 8 of 8)

- [Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection](https://mlsanity.com/paper/2409.10985/) (2024)
- [A Preliminary Study on Augmenting Speech Emotion Recognition using a Diffusion Model](https://mlsanity.com/paper/2305.11413/) (2023)
- [A Survey of Deep Learning for Group-level Emotion Recognition](https://mlsanity.com/paper/2408.15276/) (2024)
- [The Voice of Equity: A Systematic Evaluation of Bias Mitigation Techniques for Speech-Based Cognitive Impairment Detection Across Architectures and Demographics](https://mlsanity.com/paper/2601.16989/) (2026)
- [Cross-Language Speech Emotion Recognition Using Multimodal Dual Attention Transformers](https://mlsanity.com/paper/2306.13804/) (2023)
- [MoTAS: MoE-Guided Feature Selection from TTS-Augmented Speech for Enhanced Multimodal Alzheimer's Early Screening](https://mlsanity.com/paper/2508.20513/) (2025)
- [Improving Speech Emotion Recognition with Mutual Information Regularized Generative Model](https://mlsanity.com/paper/2510.10078/) (2025)
- [Synthetic Speech, Real Signal: Paralinguistic Preservation and Cross-Lingual Augmentation via Voice Cloning](https://mlsanity.com/paper/2607.22304/) (2026)

## Abstract (arXiv)

Despite advances in deep learning, current state-of-the-art speech emotion recognition (SER) systems still have poor performance due to a lack of speech emotion datasets. This paper proposes augmenting SER systems with synthetic emotional speech generated by an end-to-end text-to-speech (TTS) system based on an extended Tacotron architecture. The proposed TTS system includes encoders for speaker and emotion embeddings, a sequence-to-sequence text generator for creating Mel-spectrograms, and a WaveRNN to generate audio from the Mel-spectrograms. Extensive experiments show that the quality of the generated emotional speech can significantly improve SER performance on multiple datasets, as demonstrated by a higher mean opinion score (MOS) compared to the baseline. The generated samples were also effective at augmenting SER performance.