mlsanityData to 7 Oct 2026

SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking

arXiv 2402.02285 ↗First posted 3 Feb 2024cs.CL cs.AI cs.LGcited by 17

On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis

Abstract (arXiv)

In-context learning with Large Language Models (LLMs) has emerged as a promising avenue of research in Dialog State Tracking (DST). However, the best-performing in-context learning methods involve retrieving and adding similar examples to the prompt, requiring access to labeled training data. Procuring such training data for a wide range of domains and applications is time-consuming, expensive, and, at times, infeasible. While zero-shot learning requires no training data, it significantly lags behind the few-shot setup. Thus, `textitCan we efficiently generate synthetic data for any dialogue schema to enable few-shot prompting?' Addressing this question, we propose method, a data generation framework tailored for DST, utilizing LLMs. Our approach only requires the dialogue schema and a few hand-crafted dialogue templates to synthesize natural, coherent, and free-flowing dialogues with DST annotations. Few-shot learning using data from method results in 4-5% improvement in Joint Goal Accuracy over the zero-shot baseline on MultiWOZ 2.1 and 2.4. Remarkably, our few-shot learning approach recovers nearly 98% of the performance compared to the few-shot setup using human-annotated training data. Our synthetic data and code can be accessed at https://github.com/apple/ml-synthdst

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 33 of its 43 references that are in the map. First posted on arXiv 3 Feb 2024.

Builds on

33 of 43 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  6. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  7. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  8. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  9. 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 426
  10. 2020A Simple Language Model for Task-Oriented Dialoguecited by 331
  11. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  12. 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 215
21 more
  1. 2020MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselinescited by 201
  2. 2020TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialoguecited by 185
  3. 2016Neural Belief Tracker: Data-Driven Dialogue State Trackingcited by 142
  4. 2021Dialogue State Tracking with a Language Model using Schema-Driven Promptingcited by 94
  5. 2019Efficient Dialogue State Tracking by Selectively Overwriting Memorycited by 76
  6. 2023PLACES: Prompting Language Models for Social Conversation Synthesiscited by 72
  7. 2020Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Trackingcited by 65
  8. 2021Few-Shot Bot: Prompt-Based Learning for Dialogue Systemscited by 64
  9. 2021Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Trackingcited by 61
  10. 2022Data Curation Alone Can Stabilize In-context Learningcited by 58
  11. 2020MultiWOZ 2.3: A multi-domain task-oriented dialogue dataset enhanced with annotation corrections and co-reference annotationcited by 51
  12. 2022Controllable Dialogue Simulation with In-Context Learningcited by 42
  13. 2022InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuningcited by 39
  14. 2023ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?cited by 38
  15. 2021Annotation Inconsistency and Entity Bias in MultiWOZcited by 30
  16. 2022Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Modelscited by 26
  17. 2022LAD: Language Models as Data for Zero-Shot Dialogcited by 18
  18. 2021Few Shot Dialogue State Tracking using Meta-learningcited by 16
  19. 2022ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Modelscited by 14
  20. 2022A Dual Prompt Learning Framework for Few-Shot Dialogue State Trackingcited by 8
  21. 2021SYNERGY: Building Task Bots at Scale Using Symbolic Knowledge and Machine Teachingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 9; 2026: 3.

17 papers.

Sort:

By area:

  1. 2024Self-seeding and Multi-intent Self-instructing LLMs for Generating Intent-aware Information-Seeking dialogscited by 18
  2. 2025OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Modelscited by 7
  3. 2024DiagESC: Dialogue Synthesis for Integrating Depression Diagnosis into Emotional Support Conversationcited by 6
  4. 2024A Little Human Data Goes A Long Waycited by 6
  5. 2025Mutual Reinforcement of LLM Dialogue Synthesis and Summarization Capabilities for Few-Shot Dialogue Summarizationcited by 5
  6. 2025Synthetic Clarification and Correction Dialogues about Data-Centric Tasks -- A Teacher-Student Approachcited by 3
  7. 2024Intent-driven In-context Learning for Few-shot Dialogue State Trackingcited by 1
  8. 2025PicPersona-TOD : A Dataset for Personalizing Utterance Style in Task-Oriented Dialogue with Image Personacited by 1
  9. 2025A Typology of Synthetic Datasets for Dialogue Processing in Clinical Contextscited by 1
  10. 2025PyTOD: Programmable Task-Oriented Dialogue with Execution Feedbackcited by 1
  11. 2025PanicToCalm: A Proactive Counseling Agent for Panic Attackscited by 1
  12. 2024Beyond Ontology in Dialogue State Tracking for Goal-Oriented Chatbotcited by 0
5 more
  1. 2025MonoTODia: Translating Monologue Requests to Task-Oriented Dialoguescited by 0
  2. 2025Improving Dialogue State Tracking through Combinatorial Search for In-Context Examplescited by 0
  3. 2026ATOD: An Evaluation Framework and Benchmark for Agentic Task-Oriented Dialogue Systemscited by 0
  4. 2026On Emergences of Non-Classical Statistical Characteristics in Classical Neural Networkscited by 0
  5. 2026The Significance of Style Diversity in Annotation-Free Synthetic Data Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Large Language Models as Zero-shot Dialogue State Tracker through Function Calling
  2. 2024Enhancing Dialogue State Tracking Models through LLM-backed User-Agents Simulation
  3. 2023Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking
  4. 2024Inference is All You Need: Self Example Retriever for Cross-domain Dialogue State Tracking with ChatGPT
  5. 2023Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking
  6. 2024A Two-dimensional Zero-shot Dialogue State Tracking Evaluation Method using GPT-4
  7. 2024Benchmarks Underestimate the Readiness of Multi-lingual Dialogue Agents
  8. 2024DiaSynth: Synthetic Dialogue Generation Framework for Low Resource Dialogue Applications