mlsanityData to 14 Oct 2026

MELD-ST: An Emotion-aware Speech Translation Dataset

arXiv 2405.13233 ↗First posted 21 May 2024cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Emotion plays a crucial role in human conversation. This paper underscores the significance of considering emotion in speech translation. We present the MELD-ST dataset for the emotion-aware speech translation task, comprising English-to-Japanese and English-to-German language pairs. Each language pair includes about 10,000 utterances annotated with emotion labels from the MELD dataset. Baseline experiments using the SeamlessM4T model on the dataset indicate that fine-tuning with emotion labels can enhance translation performance in some settings, highlighting the need for further research in emotion-aware speech translation systems.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 14 of its 26 references that are in the map. First posted on arXiv 21 May 2024.

Builds on

14 of 26 references.

References · 14

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  3. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  4. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  5. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  6. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  7. 2021Direct speech-to-speech translation with discrete unitscited by 149
  8. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 98
  9. 2020CTC-Segmentation of Large Corpora for German End-to-end Speech Recognitioncited by 76
  10. 2020Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpuscited by 57
  11. 2022GigaST: A 10,000-hour Pseudo Speech Translation Corpuscited by 20
  12. 2020MuST-Cinema: a Speech-to-Subtitles corpuscited by 17
2 more
  1. 2020Breeding Gender-aware Direct Speech Translation Systemscited by 16
  2. 2023Towards Speech Dialogue Translation Mediating Speakers of Different Languagescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 3.

4 papers.

  1. 2024Speech is More Than Words: Do Speech-to-Text Translation Systems Leverage Prosody?cited by 13
  2. 2026STEB: A Speech-to-Speech Translation Expressiveness Benchmark for Evaluating Beyond Translation Fidelitycited by 1
  3. 2026Benchmarking Speech-to-Speech Translation Modelscited by 0
  4. 2026Kraken: LLM-based Speech-to-Speech Translation via Low-bitrate VQ and Dual-path Source Conditioningcited by 0