mlsanityData to 7 Oct 2026

Gaining Insights into Unrecognized User Utterances in Task-Oriented Dialog Systems

arXiv 2204.05158 ↗First posted 11 Apr 2022cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

The rapidly growing market demand for automatic dialogue agents capable of goal-oriented behavior has caused many tech-industry leaders to invest considerable efforts into task-oriented dialog systems. The success of these systems is highly dependent on the accuracy of their intent identification -- the process of deducing the goal or meaning of the user's request and mapping it to one of the known intents for further processing. Gaining insights into unrecognized utterances -- user requests the systems fail to attribute to a known intent -- is therefore a key process in continuous improvement of goal-oriented dialog systems. We present an end-to-end pipeline for processing unrecognized user utterances, deployed in a real-world, commercial task-oriented dialog system, including a specifically-tailored clustering algorithm, a novel approach to cluster representative extraction, and cluster naming. We evaluated the proposed components, demonstrating their benefits in the analysis of unrecognized user requests.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 11 of its 36 references that are in the map. First posted on arXiv 11 Apr 2022.

Builds on

11 of 36 references.

References · 11

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2018Universal Sentence Encodercited by 485
  4. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
  5. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  6. 2019Benchmarking Natural Language Understanding Services for building Conversational Agentscited by 166
  7. 2017A Survey on Dialogue Systems: Recent Advances and New Frontierscited by 162
  8. 2019Challenges in Building Intelligent Open-domain Dialog Systemscited by 147
  9. 2018Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialogcited by 124
  10. 2018Fair and Diverse DPP-based Data Summarizationcited by 39
  11. 2019Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialogcited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 0; 2025: 0; 2026: 1.

6 papers.

  1. 2023Unveiling Safety Vulnerabilities of Large Language Modelscited by 26
  2. 2023ChatGPT as your Personal Data Scientistcited by 22
  3. 2022Benchmark Data and Evaluation Framework for Intent Discovery Around COVID-19 Vaccine Hesitancycited by 4
  4. 2023Reliable and Interpretable Drift Detection in Streams of Short Textscited by 1
  5. 2023Introducing "Forecast Utterance" for Conversational Data Sciencecited by 1
  6. 2026When Words Divide: Diachronic Ideological Polarization in Political Discourse on Social Mediacited by 0