# Structure Extraction in Task-Oriented Dialogues with Slot Clustering

arXiv: https://arxiv.org/abs/2203.00073 · HTML: https://mlsanity.com/paper/2203.00073/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 71 references that are in the map.
- First posted on arXiv 28 Feb 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues](https://mlsanity.com/paper/1605.06069/) (2016)
- [Learned in Translation: Contextualized Word Vectors](https://mlsanity.com/paper/1708.00107/) (2017)
- [Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context](https://mlsanity.com/paper/1911.10484/) (2019)
- [Rasa: Open Source Language Understanding and Dialogue Management](https://mlsanity.com/paper/1712.05181/) (2017)
- [Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models](https://mlsanity.com/paper/1902.08858/) (2019)
- [Structured Attention for Unsupervised Dialogue Structure Induction](https://mlsanity.com/paper/2009.08552/) (2020)
- [Unsupervised Dialog Structure Learning](https://mlsanity.com/paper/1904.03736/) (2019)
- [Conversation Graph: Data Augmentation, Training and Evaluation for Non-Deterministic Dialogue Management](https://mlsanity.com/paper/2010.15411/) (2020)

## Built on by (most cited first, 8 of 8)

- [Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence Embeddings for Automatic Dialog Flow Extraction](https://mlsanity.com/paper/2410.18481/) (2024)
- [Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation](https://mlsanity.com/paper/2306.04724/) (2023)
- [Dialogue Term Extraction using Transfer Learning and Topological Data Analysis](https://mlsanity.com/paper/2208.10448/) (2022)
- [Using Domain Knowledge to Guide Dialog Structure Induction via Neural Probabilistic Soft Logic](https://mlsanity.com/paper/2403.17853/) (2024)
- [Turning Conversations into Workflows: A Framework to Extract and Evaluate Dialog Workflows for Service AI Agents](https://mlsanity.com/paper/2502.17321/) (2025)
- [Transforming Slot Schema Induction with Generative Dialogue State Inference](https://mlsanity.com/paper/2408.01638/) (2024)
- [Local Topology Measures of Contextual Language Model Latent Spaces With Applications to Dialogue Term Extraction](https://mlsanity.com/paper/2408.03706/) (2024)
- [Generative Induction of Dialogue Task Schemas with Streaming Refinement and Simulated Interactions](https://mlsanity.com/paper/2504.18474/) (2025)

## Abstract (arXiv)

Extracting structure information from dialogue data can help us better understand user and system behaviors. In task-oriented dialogues, dialogue structure has often been considered as transition graphs among dialogue states. However, annotating dialogue states manually is expensive and time-consuming. In this paper, we propose a simple yet effective approach for structure extraction in task-oriented dialogues. We first detect and cluster possible slot tokens with a pre-trained model to approximate dialogue ontology for a target domain. Then we track the status of each identified token group and derive a state transition structure. Empirical results show that our approach outperforms unsupervised baseline models by far in dialogue structure extraction. In addition, we show that data augmentation based on extracted structures enriches the surface formats of training data and can achieve a significant performance boost in dialogue response generation.