# MELD-ST: An Emotion-aware Speech Translation Dataset

arXiv: https://arxiv.org/abs/2405.13233 · HTML: https://mlsanity.com/paper/2405.13233/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 21 May 2024.

## Builds on (most cited first, 14 of 14)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio](https://mlsanity.com/paper/2106.06909/) (2021)
- [AudioPaLM: A Large Language Model That Can Speak and Listen](https://mlsanity.com/paper/2306.12925/) (2023)
- [Direct speech-to-speech translation with discrete units](https://mlsanity.com/paper/2107.05604/) (2021)
- [EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis](https://mlsanity.com/paper/2308.05725/) (2023)
- [CTC-Segmentation of Large Corpora for German End-to-end Speech Recognition](https://mlsanity.com/paper/2007.09127/) (2020)
- [Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpus](https://mlsanity.com/paper/2006.05754/) (2020)
- [GigaST: A 10,000-hour Pseudo Speech Translation Corpus](https://mlsanity.com/paper/2204.03939/) (2022)
- [MuST-Cinema: a Speech-to-Subtitles corpus](https://mlsanity.com/paper/2002.10829/) (2020)
- [Breeding Gender-aware Direct Speech Translation Systems](https://mlsanity.com/paper/2012.04955/) (2020)
- [Towards Speech Dialogue Translation Mediating Speakers of Different Languages](https://mlsanity.com/paper/2305.09210/) (2023)

## Built on by (most cited first, 4 of 4)

- [Speech is More Than Words: Do Speech-to-Text Translation Systems Leverage Prosody?](https://mlsanity.com/paper/2410.24019/) (2024)
- [STEB: A Speech-to-Speech Translation Expressiveness Benchmark for Evaluating Beyond Translation Fidelity](https://mlsanity.com/paper/2606.25529/) (2026)
- [Benchmarking Speech-to-Speech Translation Models](https://mlsanity.com/paper/2606.03241/) (2026)
- [Kraken: LLM-based Speech-to-Speech Translation via Low-bitrate VQ and Dual-path Source Conditioning](https://mlsanity.com/paper/2609.13045/) (2026)

## Abstract (arXiv)

Emotion plays a crucial role in human conversation. This paper underscores the significance of considering emotion in speech translation. We present the MELD-ST dataset for the emotion-aware speech translation task, comprising English-to-Japanese and English-to-German language pairs. Each language pair includes about 10,000 utterances annotated with emotion labels from the MELD dataset. Baseline experiments using the SeamlessM4T model on the dataset indicate that fine-tuning with emotion labels can enhance translation performance in some settings, highlighting the need for further research in emotion-aware speech translation systems.