# Sentiment Adaptive End-to-End Dialog Systems

arXiv: https://arxiv.org/abs/1804.10731 · HTML: https://mlsanity.com/paper/1804.10731/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2018.

## Built on by (most cited first, 17 of 17)

- [Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog](https://mlsanity.com/paper/1907.00456/) (2019)
- [Human-centric Dialog Training via Offline Reinforcement Learning](https://mlsanity.com/paper/2010.05848/) (2020)
- [Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes](https://mlsanity.com/paper/2109.08828/) (2021)
- [EmoWOZ: A Large-Scale Corpus and Labelling Scheme for Emotion Recognition in Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2109.04919/) (2021)
- [A Mixture-of-Expert Approach to RL-based Dialogue Management](https://mlsanity.com/paper/2206.00059/) (2022)
- [Empathetic Response Generation through Graph-based Multi-hop Reasoning on Emotional Causality](https://mlsanity.com/paper/2110.04614/) (2021)
- [A Student-Teacher Architecture for Dialog Domain Adaptation under the Meta-Learning Setting](https://mlsanity.com/paper/2104.02689/) (2021)
- [Task-oriented Dialogue Systems: performance vs. quality-optima, a review](https://mlsanity.com/paper/2112.11176/) (2021)
- [Are Current Task-oriented Dialogue Systems Able to Satisfy Impolite Users?](https://mlsanity.com/paper/2210.12942/) (2022)
- [Offline Reinforcement Learning for Mixture-of-Expert Dialogue Management](https://mlsanity.com/paper/2302.10850/) (2023)
- [FaceChat: An Emotion-Aware Face-to-face Dialogue Framework](https://mlsanity.com/paper/2303.07316/) (2023)
- [From Chatter to Matter: Addressing Critical Steps of Emotion Recognition Learning in Task-oriented Dialogue](https://mlsanity.com/paper/2308.12648/) (2023)
- [Using Chatbots to Teach Languages](https://mlsanity.com/paper/2208.00376/) (2022)
- [Trust in Language Grounding: a new AI challenge for human-robot teams](https://mlsanity.com/paper/2209.02066/) (2022)
- [External Knowledge Selection with Weighted Negative Sampling in Knowledge-grounded Task-oriented Dialogue Systems](https://mlsanity.com/paper/2209.02251/) (2022)
- [Reevaluating Data Partitioning for Emotion Detection in EmoWOZ](https://mlsanity.com/paper/2303.13364/) (2023)
- [Synthesizing Sentiment-Controlled Feedback For Multimodal Text and Image Data](https://mlsanity.com/paper/2402.07640/) (2024)

## Abstract (arXiv)

End-to-end learning framework is useful for building dialog systems for its simplicity in training and efficiency in model updating. However, current end-to-end approaches only consider user semantic inputs in learning and under-utilize other user information. Therefore, we propose to include user sentiment obtained through multimodal information (acoustic, dialogic and textual), in the end-to-end learning framework to make systems more user-adaptive and effective. We incorporated user sentiment information in both supervised and reinforcement learning settings. In both settings, adding sentiment information reduced the dialog length and improved the task success rate on a bus information search task. This work is the first attempt to incorporate multimodal user information in the adaptive end-to-end dialog system training framework and attained state-of-the-art performance.