# Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset

arXiv: https://arxiv.org/abs/1909.05358 · HTML: https://mlsanity.com/paper/1909.05358/

## Key facts

- Cited by 140 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2019.

## Built on by (most cited first, 30 of 140)

- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage](https://mlsanity.com/paper/2208.03188/) (2022)
- [Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System](https://mlsanity.com/paper/2109.14739/) (2021)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [KLUE: Korean Language Understanding Evaluation](https://mlsanity.com/paper/2105.09680/) (2021)
- [GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection](https://mlsanity.com/paper/2111.14592/) (2021)
- [Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.00783/) (2021)
- [Few-Shot Bot: Prompt-Based Learning for Dialogue Systems](https://mlsanity.com/paper/2110.08118/) (2021)
- [Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmark](https://mlsanity.com/paper/2405.08355/) (2024)
- [Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey](https://mlsanity.com/paper/2503.22458/) (2025)
- [BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling](https://mlsanity.com/paper/2106.02787/) (2021)
- [Dialog Inpainting: Turning Documents into Dialogs](https://mlsanity.com/paper/2205.09073/) (2022)
- [WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting](https://mlsanity.com/paper/2405.00823/) (2024)
- [Rethinking Explainability as a Dialogue: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01875/) (2022)
- [PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistants](https://mlsanity.com/paper/2506.09902/) (2025)
- [Recent Advances of Foundation Language Models-based Continual Learning: A Survey](https://mlsanity.com/paper/2405.18653/) (2024)
- [ToolTalk: Evaluating Tool-Usage in a Conversational Setting](https://mlsanity.com/paper/2311.10775/) (2023)
- [Neural Approaches to Conversational Information Retrieval](https://mlsanity.com/paper/2201.05176/) (2022)
- [InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning](https://mlsanity.com/paper/2205.12673/) (2022)
- [GEMv2: Multilingual NLG Benchmarking in a Single Line of Code](https://mlsanity.com/paper/2206.11249/) (2022)
- [TWEETSUMM -- A Dialog Summarization Dataset for Customer Service](https://mlsanity.com/paper/2111.11894/) (2021)
- [SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation](https://mlsanity.com/paper/2209.06664/) (2022)
- [Large Language Models as Zero-shot Dialogue State Tracker through Function Calling](https://mlsanity.com/paper/2402.10466/) (2024)
- [AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models](https://mlsanity.com/paper/2102.05126/) (2021)
- [Annotation Inconsistency and Entity Bias in MultiWOZ](https://mlsanity.com/paper/2105.14150/) (2021)
- [Fusing task-oriented and open-domain dialogues in conversational agents](https://mlsanity.com/paper/2109.04137/) (2021)
- [SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understanding](https://mlsanity.com/paper/2209.06638/) (2022)
- [Position: Measure Dataset Diversity, Don't Just Claim It](https://mlsanity.com/paper/2407.08188/) (2024)
- [CoCoA-MT: A Dataset and Benchmark for Contrastive Controlled MT with Application to Formality](https://mlsanity.com/paper/2205.04022/) (2022)
- [Pretraining the Noisy Channel Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2103.10518/) (2021)

## Abstract (arXiv)

A significant barrier to progress in data-driven approaches to building dialog systems is the lack of high quality, goal-oriented conversational data. To help satisfy this elementary requirement, we introduce the initial release of the Taskmaster-1 dataset which includes 13,215 task-based dialogs comprising six domains. Two procedures were used to create this collection, each with unique advantages. The first involves a two-person, spoken "Wizard of Oz" (WOz) approach in which trained agents and crowdsourced workers interact to complete the task while the second is "self-dialog" in which crowdsourced workers write the entire dialog themselves. We do not restrict the workers to detailed scripts or to a small knowledge base and hence we observe that our dataset contains more realistic and diverse conversations in comparison to existing datasets. We offer several baseline models including state of the art neural seq2seq architectures with benchmark performance as well as qualitative human evaluations. Dialogs are labeled with API calls and arguments, a simple and cost effective approach which avoids the requirement of complex annotation schema. The layer of abstraction between the dialog model and the service provider API allows for a given model to interact with multiple services that provide similar functionally. Finally, the dataset will evoke interest in written vs. spoken language, discourse patterns, error handling and other linguistic phenomena related to dialog system research, development and design.