mlsanityData to 7 Oct 2026

Learning Emotional Representations from Imbalanced Speech Data for Speech Emotion Recognition and Emotional Text-to-Speech

arXiv 2306.05709 ↗First posted 9 Jun 2023eess.AS cs.CL cs.SDcited by 4

Not on the map. Why?

Abstract (arXiv)

Effective speech emotional representations play a key role in Speech Emotion Recognition (SER) and Emotional Text-To-Speech (TTS) tasks. However, emotional speech samples are more difficult and expensive to acquire compared with Neutral style speech, which causes one issue that most related works unfortunately neglect: imbalanced datasets. Models might overfit to the majority Neutral class and fail to produce robust and effective emotional representations. In this paper, we propose an Emotion Extractor to address this issue. We use augmentation approaches to train the model and enable it to extract effective and generalizable emotional representations from imbalanced datasets. Our empirical results show that (1) for the SER task, the proposed Emotion Extractor surpasses the state-of-the-art baseline on three imbalanced datasets; (2) the produced representations from our Emotion Extractor benefit the TTS model, and enable it to synthesize more expressive speech.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 21 of its 31 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

21 of 31 references.

References · 21

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  7. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  8. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 119
  9. 2018The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systemscited by 61
  10. 2018Robust and fine-grained prosody control of end-to-end speech synthesiscited by 48
  11. 2021MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognitioncited by 38
  12. 2022An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Eracited by 32
9 more
  1. 2022MsEmoTTS: Multi-scale emotion transfer, prediction, and control for emotional speech synthesiscited by 29
  2. 2020Low-resource expressive text-to-speech using data augmentationcited by 26
  3. 2020TTS-by-TTS: TTS-driven Data Augmentation for Fast and High-Quality Speech Synthesiscited by 15
  4. 2020Augmenting Generative Adversarial Networks for Speech Emotion Recognitioncited by 11
  5. 2022Cross-speaker style transfer for text-to-speech using data augmentationcited by 9
  6. 2023Fine-grained Emotional Control of Text-To-Speech: Learning To Rank Inter- And Intra-Class Emotion Intensitiescited by 5
  7. 2022Contrastive-mixup learning for improved speaker verificationcited by 4
  8. 2022Improving Speech Emotion Recognition Through Focus and Calibration Attention Mechanismscited by 3
  9. 2022Generative Data Augmentation Guided by Triplet Loss for Speech Emotion Recognitioncited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 2.

4 papers.

  1. 2024EmoSphere-TTS: Emotional Style and Intensity Modeling via Spherical Emotion Vector for Controllable Emotional Text-to-Speechcited by 23
  2. 2023EMOCONV-DIFF: Diffusion-based Speech Emotion Conversion for Non-parallel and In-the-wild Datacited by 7
  3. 2025UDDETTS: Unifying Discrete and Dimensional Emotions for Controllable Emotional Text-to-Speechcited by 2
  4. 2025Seeing is Believing: Emotion-Aware Audio-Visual Language Modeling for Expressive Speech Generationcited by 0