mlsanityData to 14 Oct 2026

Speech Emotion Diarization: Which Emotion Appears When?

arXiv 2306.12991 ↗First posted 22 Jun 2023cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Speech Emotion Recognition (SER) typically relies on utterance-level solutions. However, emotions conveyed through speech should be considered as discrete speech events with definite temporal boundaries, rather than attributes of the entire utterance. To reflect the fine-grained nature of speech emotions, we propose a new task: Speech Emotion Diarization (SED). Just as Speaker Diarization answers the question of "Who speaks when?", Speech Emotion Diarization answers the question of "Which emotion appears when?". To facilitate the evaluation of the performance and establish a common benchmark for researchers, we introduce the Zaion Emotion Dataset (ZED), an openly accessible speech emotion dataset that includes non-acted emotions recorded in real-life conditions, along with manually-annotated boundaries of emotion segments within the utterance. We provide competitive baselines and open-source the code and the pre-trained models.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 12 of its 39 references that are in the map. First posted on arXiv 22 Jun 2023.

Builds on

12 of 39 references.

References · 12

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 406
  6. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 121
  7. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  8. 2019SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wildcited by 62
  9. 2018The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systemscited by 61
  10. 2022Boosting Self-Supervised Embeddings for Speech Enhancementcited by 24
  11. 2022Speaker Normalization for Self-supervised Speech Emotion Recognitioncited by 14
  12. 2017Emotion Recognition From Speech With Recurrent Neural Networkscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 6; 2026: 2.

13 papers.

Sort:

By area:

  1. 2024Open-Source Conversational AI with SpeechBrain 1.0cited by 56
  2. 2024ED-TTS: Multi-Scale Emotion Modeling using Cross-Domain Emotion Diarization for Emotional Speech Synthesiscited by 9
  3. 2024SKILL: Similarity-aware Knowledge distILLation for Speech Self-Supervised Learningcited by 5
  4. 2024What Are They Doing? Joint Audio-Speech Co-Reasoningcited by 4
  5. 2025Segmentation-Variant Codebooks for Preservation of Paralinguistic and Prosodic Informationcited by 4
  6. 2024Cross-Lingual Speech Emotion Recognition: Humans vs. Self-Supervised Modelscited by 3
  7. 2025Unsupervised Speech Segmentation: A General Approach Using Speech Language Modelscited by 1
  8. 2025StutterCut: Uncertainty-Guided Normalised Cut for Dysfluency Segmentationcited by 1
  9. 2025Maestro-EVC: Controllable Emotional Voice Conversion Guided by References and Explicit Prosodycited by 1
  10. 2025Enhancing Speech Emotion Recognition Leveraging Aligning Timestamps of ASR Transcripts and Speaker Diarizationcited by 0
  11. 2025SEER: The Span-based Emotion Evidence Retrieval Benchmarkcited by 0
  12. 2026EmoTransCap: Dataset and Pipeline for Emotion Transition-Aware Speech Captioning in Discoursescited by 0
1 more
  1. 2026Finding Emotions Where They Belong: Rethinking Audio Emotion Recognition through Masked Temporal Affective Groundingcited by 0