mlsanityData to 14 Oct 2026

Rating Prediction in Conversational Task Assistants with Behavioral and Conversational-Flow Features

arXiv 2309.11307 ↗First posted 20 Sep 2023cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Predicting the success of Conversational Task Assistants (CTA) can be critical to understand user behavior and act accordingly. In this paper, we propose TB-Rater, a Transformer model which combines conversational-flow features with user behavior features for predicting user ratings in a CTA scenario. In particular, we use real human-agent conversations and ratings collected in the Alexa TaskBot challenge, a novel multimodal and multi-turn conversational context. Our results show the advantages of modeling both the conversational-flow and behavioral aspects of the conversation in a single model for offline rating prediction. Additionally, an analysis of the CTA-specific behavioral features brings insights into this setting and can be used to bootstrap future systems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 31 references that are in the map. First posted on arXiv 20 Sep 2023.

Builds on

15 of 31 references.

References · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  5. 2020A Simple Language Model for Task-Oriented Dialoguecited by 332
  6. 2020UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2cited by 111
  7. 2018Conversational AI: The Science Behind the Alexa Prizecited by 63
  8. 2022Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agentscited by 62
  9. 2022Conversational Information Seekingcited by 37
  10. 2019An End-to-End Dialogue State Tracking System with Machine Reading Comprehension and Wide & Deep Classificationcited by 20
  11. 2020IART: Intent-aware Response Ranking with Transformers in Information-seeking Conversation Systemscited by 16
  12. 2022Alexa, Let's Work Together: Introducing the First Alexa Prize TaskBot Challenge on Conversational Task Assistancecited by 15
3 more
  1. 2020Offline and Online Satisfaction Prediction in Open-Domain Conversational Systemscited by 12
  2. 2022Task2Dial: A Novel Task and Dataset for Commonsense enhanced Task-based Dialogue Grounded in Documentscited by 7
  3. 2022Understanding User Satisfaction with Task-oriented Dialogue Systemscited by 7

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023TWIZ-v2: The Wizard of Multimodal Conversational-Stimuluscited by 2