# Task-oriented Dialogue Systems: performance vs. quality-optima, a review

arXiv: https://arxiv.org/abs/2112.11176 · HTML: https://mlsanity.com/paper/2112.11176/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 133 references that are in the map.
- First posted on arXiv 21 Dec 2021.

## Builds on (most cited first, 22 of 22)

- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [A Neural Conversational Model](https://mlsanity.com/paper/1506.05869/) (2015)
- [Neural Belief Tracker: Data-Driven Dialogue State Tracking](https://mlsanity.com/paper/1606.03777/) (2016)
- [Survey on Evaluation Methods for Dialogue Systems](https://mlsanity.com/paper/1905.04071/) (2019)
- [Recent Advances and Challenges in Task-oriented Dialog System](https://mlsanity.com/paper/2003.07490/) (2020)
- [Global-Locally Self-Attentive Dialogue State Tracker](https://mlsanity.com/paper/1805.09655/) (2018)
- [Towards Unified Dialogue System Evaluation: A Comprehensive Analysis of Current Evaluation Protocols](https://mlsanity.com/paper/2006.06110/) (2020)
- [Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning](https://mlsanity.com/paper/1702.03274/) (2017)
- [On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systems](https://mlsanity.com/paper/1605.07669/) (2016)
- [An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking](https://mlsanity.com/paper/1805.01555/) (2018)
- [A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions](https://mlsanity.com/paper/1906.00500/) (2019)
- [Flexible End-to-End Dialogue System for Knowledge Grounded Conversation](https://mlsanity.com/paper/1709.04264/) (2017)
- [Affective Neural Response Generation](https://mlsanity.com/paper/1709.03968/) (2017)
- [Sentiment Adaptive End-to-End Dialog Systems](https://mlsanity.com/paper/1804.10731/) (2018)
- [Sentiment Analysis of Twitter Data: A Survey of Techniques](https://mlsanity.com/paper/1601.06971/) (2016)
- [Evaluating Quality of Chatbots and Intelligent Conversational Agents](https://mlsanity.com/paper/1704.04579/) (2017)
- [Learning from Real Users: Rating Dialogue Success with Neural Networks for Reinforcement Learning in Spoken Dialogue Systems](https://mlsanity.com/paper/1508.03386/) (2015)

## Built on by (most cited first, 3 of 3)

- [A Survey on Recent Advances in Conversational Data Generation](https://mlsanity.com/paper/2405.13003/) (2024)
- [A Machine Learning Approach to Classifying Construction Cost Documents into the International Construction Measurement Standard](https://mlsanity.com/paper/2211.07705/) (2022)
- [Curricular Transfer Learning for Sentence Encoded Tasks](https://mlsanity.com/paper/2308.01849/) (2023)

## Abstract (arXiv)

Task-oriented dialogue systems (TODS) are continuing to rise in popularity as various industries find ways to effectively harness their capabilities, saving both time and money. However, even state-of-the-art TODS are not yet reaching their full potential. TODS typically have a primary design focus on completing the task at hand, so the metric of task-resolution should take priority. Other conversational quality attributes that may point to the success, or otherwise, of the dialogue, may be ignored. This can cause interactions between human and dialogue system that leave the user dissatisfied or frustrated. This paper explores the literature on evaluative frameworks of dialogue systems and the role of conversational quality attributes in dialogue systems, looking at if, how, and where they are utilised, and examining their correlation with the performance of the dialogue system.