# Infusing Emotions into Task-oriented Dialogue Systems: Understanding, Management, and Generation

arXiv: https://arxiv.org/abs/2408.02417 · HTML: https://mlsanity.com/paper/2408.02417/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 5 Aug 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [Few-shot Natural Language Generation for Task-Oriented Dialog](https://mlsanity.com/paper/2002.12328/) (2020)
- [Adding Chit-Chat to Enhance Task-Oriented Dialogues](https://mlsanity.com/paper/2010.12757/) (2020)
- [Neural User Simulation for Corpus-based Policy Optimisation for Spoken Dialogue Systems](https://mlsanity.com/paper/1805.06966/) (2018)
- [A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learning](https://mlsanity.com/paper/2202.13675/) (2022)
- [Fusing task-oriented and open-domain dialogues in conversational agents](https://mlsanity.com/paper/2109.04137/) (2021)
- [Domain-independent User Simulation with Transformers for Task-oriented Dialogue Systems](https://mlsanity.com/paper/2106.08838/) (2021)
- [GenTUS: Simulating User Behaviour and Language in Task-oriented Dialogues with Generative Transformers](https://mlsanity.com/paper/2208.10817/) (2022)
- [ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format](https://mlsanity.com/paper/2211.17148/) (2022)
- [UniSA: Unified Generative Framework for Sentiment Analysis](https://mlsanity.com/paper/2309.01339/) (2023)
- [Robust Dialogue State Tracking with Weak Supervision and Sparse Data](https://mlsanity.com/paper/2202.03354/) (2022)
- [Dynamic Dialogue Policy for Continual Reinforcement Learning](https://mlsanity.com/paper/2204.05928/) (2022)
- [EmoUS: Simulating User Emotions in Task-Oriented Dialogues](https://mlsanity.com/paper/2306.01579/) (2023)
- [Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performance](https://mlsanity.com/paper/2109.04349/) (2021)
- [DialCrowd 2.0: A Quality-Focused Dialog System Crowdsourcing Toolkit](https://mlsanity.com/paper/2207.12551/) (2022)
- [Dialogue Quality and Emotion Annotations for Customer Support Conversations](https://mlsanity.com/paper/2311.13910/) (2023)
- [Dialogue Evaluation with Offline Reinforcement Learning](https://mlsanity.com/paper/2209.00876/) (2022)
- [A Unified Approach to Emotion Detection and Task-Oriented Dialogue Modeling](https://mlsanity.com/paper/2401.13789/) (2024)

## Built on by (most cited first, 5 of 5)

- [PicPersona-TOD : A Dataset for Personalizing Utterance Style in Task-Oriented Dialogue with Image Persona](https://mlsanity.com/paper/2504.17390/) (2025)
- [Sentipolis: Emotion-Aware Agents for Social Simulations](https://mlsanity.com/paper/2601.18027/) (2026)
- [Hear2Act: Benchmarking When Prosody Should Change What an Assistant Does](https://mlsanity.com/paper/2608.19515/) (2026)
- [Text-to-SQL Task-oriented Dialogue Ontology Construction](https://mlsanity.com/paper/2507.23358/) (2025)
- [Evaluating LLM-Generated Versus Human-Authored Responses in Role-Play Dialogues](https://mlsanity.com/paper/2509.17694/) (2025)

## Abstract (arXiv)

Emotions are indispensable in human communication, but are often overlooked in task-oriented dialogue (ToD) modelling, where the task success is the primary focus. While existing works have explored user emotions or similar concepts in some ToD tasks, none has so far included emotion modelling into a fully-fledged ToD system nor conducted interaction with human or simulated users. In this work, we incorporate emotion into the complete ToD processing loop, involving understanding, management, and generation. To this end, we extend the EmoWOZ dataset (Feng et al., 2022) with system affective behaviour labels. Through interactive experimentation involving both simulated and human users, we demonstrate that our proposed framework significantly enhances the user's emotional experience as well as the task success.