# Generative Induction of Dialogue Task Schemas with Streaming Refinement and Simulated Interactions

arXiv: https://arxiv.org/abs/2504.18474 · HTML: https://mlsanity.com/paper/2504.18474/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 57 references that are in the map.
- First posted on arXiv 25 Apr 2025.

## Builds on (most cited first, 30 of 34)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [Time Travel in LLMs: Tracing Data Contamination in Large Language Models](https://mlsanity.com/paper/2308.08493/) (2023)
- [Benchmark Data Contamination of Large Language Models: A Survey](https://mlsanity.com/paper/2406.04244/) (2024)
- [MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluation](https://mlsanity.com/paper/2104.00773/) (2021)
- [Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2005.00891/) (2020)
- [SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems](https://mlsanity.com/paper/2110.06800/) (2021)
- [Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructions](https://mlsanity.com/paper/2010.10216/) (2020)
- [Investigating the Impact of Data Contamination of Large Language Models in Text-to-SQL Translation](https://mlsanity.com/paper/2402.08100/) (2024)
- [LAD: Language Models as Data for Zero-Shot Dialog](https://mlsanity.com/paper/2207.14393/) (2022)
- [NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation](https://mlsanity.com/paper/2105.14454/) (2021)
- [Show, Don't Tell: Demonstrations Outperform Descriptions for Schema-Guided Task-Oriented Dialogue](https://mlsanity.com/paper/2204.04327/) (2022)
- [Generalized Category Discovery with Large Language Models in the Loop](https://mlsanity.com/paper/2312.10897/) (2023)
- [Workflow Discovery from Dialogues in the Low Data Regime](https://mlsanity.com/paper/2205.11690/) (2022)
- [Unsupervised Slot Schema Induction for Task-oriented Dialog](https://mlsanity.com/paper/2205.04515/) (2022)
- [Dialogue State Induction Using Neural Latent Variable Models](https://mlsanity.com/paper/2008.05666/) (2020)
- [Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking](https://mlsanity.com/paper/2307.01453/) (2023)
- [Zero-Shot On-the-Fly Event Schema Induction](https://mlsanity.com/paper/2210.06254/) (2022)
- [Structure Extraction in Task-Oriented Dialogues with Slot Clustering](https://mlsanity.com/paper/2203.00073/) (2022)
- [IntentGPT: Few-shot Intent Discovery with Large Language Models](https://mlsanity.com/paper/2411.10670/) (2024)
- [N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking](https://mlsanity.com/paper/2103.00293/) (2021)
- [Diverse and Effective Synthetic Data Generation for Adaptable Zero-Shot Dialogue State Tracking](https://mlsanity.com/paper/2405.12468/) (2024)
- [A Survey of Ontology Expansion for Conversational Understanding](https://mlsanity.com/paper/2410.15019/) (2024)

## Built on by (most cited first, 2 of 2)

- [METRO: Towards Strategy Induction from Expert Dialogue Transcripts for Non-collaborative Dialogues](https://mlsanity.com/paper/2604.11427/) (2026)
- [Induce, Align, Predict: Zero-Shot Stance Detection via Cognitive Inductive Reasoning](https://mlsanity.com/paper/2506.13470/) (2025)

## Abstract (arXiv)

In task-oriented dialogue (TOD) systems, Slot Schema Induction (SSI) is essential for automatically identifying key information slots from dialogue data without manual intervention. This paper presents a novel state-of-the-art (SoTA) approach that formulates SSI as a text generation task, where a language model incrementally constructs and refines a slot schema over a stream of dialogue data. To develop this approach, we present a fully automatic LLM-based TOD simulation method that creates data with high-quality state labels for novel task domains. Furthermore, we identify issues in SSI evaluation due to data leakage and poor metric alignment with human judgment. We resolve these by creating new evaluation data using our simulation method with human guidance and correction, as well as designing improved evaluation metrics. These contributions establish a foundation for future SSI research and advance the SoTA in dialogue understanding and system development.