# Reevaluating Data Partitioning for Emotion Detection in EmoWOZ

arXiv: https://arxiv.org/abs/2303.13364 · HTML: https://mlsanity.com/paper/2303.13364/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Builds on (most cited first, 9 of 9)

- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](https://mlsanity.com/paper/1811.00405/) (2018)
- [COSMIC: COmmonSense knowledge for eMotion Identification in Conversations](https://mlsanity.com/paper/2010.02795/) (2020)
- [Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networks](https://mlsanity.com/paper/1708.04299/) (2017)
- [EmoWOZ: A Large-Scale Corpus and Labelling Scheme for Emotion Recognition in Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2109.04919/) (2021)
- [Sentiment Adaptive End-to-End Dialog Systems](https://mlsanity.com/paper/1804.10731/) (2018)
- [Towards Emotion-Aware User Simulator for Task-Oriented Dialogue](https://mlsanity.com/paper/2011.09696/) (2020)

## Abstract (arXiv)

This paper focuses on the EmoWoz dataset, an extension of MultiWOZ that provides emotion labels for the dialogues. MultiWOZ was partitioned initially for another purpose, resulting in a distributional shift when considering the new purpose of emotion recognition. The emotion tags in EmoWoz are highly imbalanced and unevenly distributed across the partitions, which causes sub-optimal performance and poor comparison of models. We propose a stratified sampling scheme based on emotion tags to address this issue, improve the dataset's distribution, and reduce dataset shift. We also introduce a special technique to handle conversation (sequential) data with many emotional tags. Using our proposed sampling method, models built upon EmoWoz can perform better, making it a more reliable resource for training conversational agents with emotional intelligence. We recommend that future researchers use this new partitioning to ensure consistent and accurate performance evaluations.