Task-oriented Dialogue Systems: performance vs. quality-optima, a review
Not on the map. Why?
Abstract (arXiv)
Task-oriented dialogue systems (TODS) are continuing to rise in popularity as various industries find ways to effectively harness their capabilities, saving both time and money. However, even state-of-the-art TODS are not yet reaching their full potential. TODS typically have a primary design focus on completing the task at hand, so the metric of task-resolution should take priority. Other conversational quality attributes that may point to the success, or otherwise, of the dialogue, may be ignored. This can cause interactions between human and dialogue system that leave the user dissatisfied or frustrated. This paper explores the literature on evaluative frameworks of dialogue systems and the role of conversational quality attributes in dialogue systems, looking at if, how, and where they are utilised, and examining their correlation with the performance of the dialogue system.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 22 of its 133 references that are in the map. First posted on arXiv 21 Dec 2021.
Builds on
22 of 133 references.
References · 22
- 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
- 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
- 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
- 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
- 2017Why We Need New Evaluation Metrics for NLGcited by 233
- 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 216
- 2015A Neural Conversational Modelcited by 196
- 2016Neural Belief Tracker: Data-Driven Dialogue State Trackingcited by 143
- 2019Survey on Evaluation Methods for Dialogue Systemscited by 126
- 2020Recent Advances and Challenges in Task-oriented Dialog Systemcited by 78
- 2018Global-Locally Self-Attentive Dialogue State Trackercited by 45
- 2020Towards Unified Dialogue System Evaluation: A Comprehensive Analysis of Current Evaluation Protocolscited by 35
10 more
- 2017Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learningcited by 32
- 2016On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systemscited by 28
- 2018An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Trackingcited by 27
- 2019A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directionscited by 26
- 2017Flexible End-to-End Dialogue System for Knowledge Grounded Conversationcited by 18
- 2017Affective Neural Response Generationcited by 17
- 2018Sentiment Adaptive End-to-End Dialog Systemscited by 17
- 2016Sentiment Analysis of Twitter Data: A Survey of Techniquescited by 16
- 2017Evaluating Quality of Chatbots and Intelligent Conversational Agentscited by 16
- 2015Learning from Real Users: Rating Dialogue Success with Neural Networks for Reinforcement Learning in Spoken Dialogue Systemscited by 8
Built on by
Citing papers per year: 2022: 1; 2023: 1; 2024: 1.
3 papers.