# Unsupervised Slot Schema Induction for Task-oriented Dialog

arXiv: https://arxiv.org/abs/2205.04515 · HTML: https://mlsanity.com/paper/2205.04515/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 74 references that are in the map.
- First posted on arXiv 9 May 2022.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering](https://mlsanity.com/paper/1611.05148/) (2016)
- [MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines](https://mlsanity.com/paper/2007.12720/) (2020)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems](https://mlsanity.com/paper/1506.08909/) (2015)
- [Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context](https://mlsanity.com/paper/1911.10484/) (2019)
- [A Discrete Hard EM Approach for Weakly Supervised Question Answering](https://mlsanity.com/paper/1909.04849/) (2019)
- [Span-based Semantic Parsing for Compositional Generalization](https://mlsanity.com/paper/2009.06040/) (2020)
- [Compound Probabilistic Context-Free Grammars for Grammar Induction](https://mlsanity.com/paper/1906.10225/) (2019)
- [Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.00783/) (2021)
- [Zero-Shot Transfer Learning for Event Extraction](https://mlsanity.com/paper/1707.01066/) (2017)
- [Simple Unsupervised Keyphrase Extraction using Sentence Embeddings](https://mlsanity.com/paper/1801.04470/) (2018)
- [Neural Language Modeling by Jointly Learning Syntax and Lexicon](https://mlsanity.com/paper/1711.02013/) (2017)
- [PMI-Masking: Principled masking of correlated spans](https://mlsanity.com/paper/2010.01825/) (2020)
- [Visually Grounded Neural Syntax Acquisition](https://mlsanity.com/paper/1906.02890/) (2019)
- [Are Pre-trained Language Models Aware of Phrases? Simple but Strong Baselines for Grammar Induction](https://mlsanity.com/paper/2002.00737/) (2020)
- [Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention](https://mlsanity.com/paper/1905.12866/) (2019)
- [Few-shot Intent Classification and Slot Filling with Retrieved Examples](https://mlsanity.com/paper/2104.05763/) (2021)
- [Asking without Telling: Exploring Latent Ontologies in Contextual Representations](https://mlsanity.com/paper/2004.14513/) (2020)
- [ConVEx: Data-Efficient and Few-Shot Slot Labeling](https://mlsanity.com/paper/2010.11791/) (2020)
- [MIDAS: A Dialog Act Annotation Scheme for Open Domain Human Machine Spoken Conversations](https://mlsanity.com/paper/1908.10023/) (2019)
- [Corpus-based Open-Domain Event Type Induction](https://mlsanity.com/paper/2109.03322/) (2021)
- [Video-aided Unsupervised Grammar Induction](https://mlsanity.com/paper/2104.04369/) (2021)
- [Probing Task-Oriented Dialogue Representation from Language Models](https://mlsanity.com/paper/2010.13912/) (2020)
- [Dialogue State Induction Using Neural Latent Variable Models](https://mlsanity.com/paper/2008.05666/) (2020)

## Built on by (most cited first, 11 of 11)

- [Workflow Discovery from Dialogues in the Low Data Regime](https://mlsanity.com/paper/2205.11690/) (2022)
- [AnyTOD: A Programmable Task-Oriented Dialog System](https://mlsanity.com/paper/2212.09939/) (2022)
- [A Survey of Ontology Expansion for Conversational Understanding](https://mlsanity.com/paper/2410.15019/) (2024)
- [Dialogue Term Extraction using Transfer Learning and Topological Data Analysis](https://mlsanity.com/paper/2208.10448/) (2022)
- [Knowledge-grounded Dialog State Tracking](https://mlsanity.com/paper/2210.06656/) (2022)
- [NatCS: Eliciting Natural Customer Support Dialogues](https://mlsanity.com/paper/2305.03007/) (2023)
- [Slot Induction via Pre-trained Language Model Probing and Multi-level Contrastive Learning](https://mlsanity.com/paper/2308.04712/) (2023)
- [Transforming Slot Schema Induction with Generative Dialogue State Inference](https://mlsanity.com/paper/2408.01638/) (2024)
- [Generative Induction of Dialogue Task Schemas with Streaming Refinement and Simulated Interactions](https://mlsanity.com/paper/2504.18474/) (2025)
- [Grounding Description-Driven Dialogue State Trackers with Knowledge-Seeking Turns](https://mlsanity.com/paper/2309.13448/) (2023)
- [Text-to-SQL Task-oriented Dialogue Ontology Construction](https://mlsanity.com/paper/2507.23358/) (2025)

## Abstract (arXiv)

Carefully-designed schemas describing how to collect and annotate dialog corpora are a prerequisite towards building task-oriented dialog systems. In practical applications, manually designing schemas can be error-prone, laborious, iterative, and slow, especially when the schema is complicated. To alleviate this expensive and time consuming process, we propose an unsupervised approach for slot schema induction from unlabeled dialog corpora. Leveraging in-domain language models and unsupervised parsing structures, our data-driven approach extracts candidate slots without constraints, followed by coarse-to-fine clustering to induce slot types. We compare our method against several strong supervised baselines, and show significant performance improvement in slot schema induction on MultiWoz and SGD datasets. We also demonstrate the effectiveness of induced schemas on downstream applications including dialog state tracking and response generation.