# LAD: Language Models as Data for Zero-Shot Dialog

arXiv: https://arxiv.org/abs/2207.14393 · HTML: https://mlsanity.com/paper/2207.14393/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 54 references that are in the map.
- First posted on arXiv 28 Jul 2022.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://mlsanity.com/paper/2104.08826/) (2021)
- [Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little](https://mlsanity.com/paper/2104.06644/) (2021)
- [Benchmarking Natural Language Understanding Services for building Conversational Agents](https://mlsanity.com/paper/1903.05566/) (2019)
- [DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue](https://mlsanity.com/paper/2009.13570/) (2020)
- [Towards Zero-Label Language Learning](https://mlsanity.com/paper/2109.09193/) (2021)
- [SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching](https://mlsanity.com/paper/2005.05298/) (2020)
- [Few-Shot Bot: Prompt-Based Learning for Dialogue Systems](https://mlsanity.com/paper/2110.08118/) (2021)
- [A User Simulator for Task-Completion Dialogues](https://mlsanity.com/paper/1612.05688/) (2016)
- [How to Build User Simulators to Train RL-based Dialog Systems](https://mlsanity.com/paper/1909.01388/) (2019)
- [STAR: A Schema-Guided Dialog Dataset for Transfer Learning](https://mlsanity.com/paper/2010.11853/) (2020)
- [Neural User Simulation for Corpus-based Policy Optimisation for Spoken Dialogue Systems](https://mlsanity.com/paper/1805.06966/) (2018)
- [Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations](https://mlsanity.com/paper/2005.08866/) (2020)
- [Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling](https://mlsanity.com/paper/2004.11727/) (2020)
- [Example-Driven Intent Prediction with Observers](https://mlsanity.com/paper/2010.08684/) (2020)
- [Domain-independent User Simulation with Transformers for Task-oriented Dialogue Systems](https://mlsanity.com/paper/2106.08838/) (2021)
- [ConVEx: Data-Efficient and Few-Shot Slot Labeling](https://mlsanity.com/paper/2010.11791/) (2020)
- [Schema-Guided Paradigm for Zero-Shot Dialog](https://mlsanity.com/paper/2106.07056/) (2021)

## Built on by (most cited first, 18 of 18)

- [PLACES: Prompting Language Models for Social Conversation Synthesis](https://mlsanity.com/paper/2302.03269/) (2023)
- [Leveraging Large Language Models in Conversational Recommender Systems](https://mlsanity.com/paper/2305.07961/) (2023)
- [Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data](https://mlsanity.com/paper/2301.05843/) (2023)
- [Faithful Persona-based Conversational Dataset Generation with Large Language Models](https://mlsanity.com/paper/2312.10007/) (2023)
- [SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting](https://mlsanity.com/paper/2305.09067/) (2023)
- [Controllable Mixed-Initiative Dialogue Generation through Prompting](https://mlsanity.com/paper/2305.04147/) (2023)
- [COKE: A Cognitive Knowledge Graph for Machine Theory of Mind](https://mlsanity.com/paper/2305.05390/) (2023)
- [SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking](https://mlsanity.com/paper/2402.02285/) (2024)
- [PersonalityChat: Conversation Distillation for Personalized Dialog Modeling with Facts and Traits](https://mlsanity.com/paper/2401.07363/) (2024)
- [Efficient Data Generation for Source-grounded Information-seeking Dialogs: A Use Case for Meeting Transcripts](https://mlsanity.com/paper/2405.01121/) (2024)
- [Diverse and Effective Synthetic Data Generation for Adaptable Zero-Shot Dialogue State Tracking](https://mlsanity.com/paper/2405.12468/) (2024)
- [Mutual Reinforcement of LLM Dialogue Synthesis and Summarization Capabilities for Few-Shot Dialogue Summarization](https://mlsanity.com/paper/2502.17328/) (2025)
- [Controllable and Diverse Data Augmentation with Large Language Model for Low-Resource Open-Domain Dialogue Generation](https://mlsanity.com/paper/2404.00361/) (2024)
- [Data Augmentation for Conversational AI](https://mlsanity.com/paper/2309.04739/) (2023)
- [Making Task-Oriented Dialogue Datasets More Natural by Synthetically Generating Indirect User Requests](https://mlsanity.com/paper/2406.07794/) (2024)
- [Generative Induction of Dialogue Task Schemas with Streaming Refinement and Simulated Interactions](https://mlsanity.com/paper/2504.18474/) (2025)
- [Comparing Data Augmentation Methods for End-to-End Task-Oriented Dialog Systems](https://mlsanity.com/paper/2406.06127/) (2024)
- [MonoTODia: Translating Monologue Requests to Task-Oriented Dialogues](https://mlsanity.com/paper/2502.17268/) (2025)

## Abstract (arXiv)

To facilitate zero-shot generalization in taskoriented dialog, this paper proposes Language Models as Data (LAD). LAD is a paradigm for creating diverse and accurate synthetic data which conveys the necessary structural constraints and can be used to train a downstream neural dialog model. LAD leverages GPT-3 to induce linguistic diversity. LAD achieves significant performance gains in zero-shot settings on intent prediction (+15%), slot filling (+31.4 F-1) and next action prediction (+11 F1). Furthermore, an interactive human evaluation shows that training with LAD is competitive with training on human dialogs. LAD is open-sourced, with the code and data available at https://github.com/Shikib/lad.