mlsanityData to 7 Oct 2026

MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines

arXiv 2007.12720 ↗First posted 10 Jul 2020cs.CL cs.AIcited by 201

Not on the map. Why?

Cited by 22 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

MultiWOZ is a well-known task-oriented dialogue dataset containing over 10,000 annotated dialogues spanning 8 domains. It is extensively used as a benchmark for dialogue state tracking. However, recent works have reported presence of substantial noise in the dialogue state annotations. MultiWOZ 2.1 identified and fixed many of these erroneous annotations and user utterances, resulting in an improved version of this dataset. This work introduces MultiWOZ 2.2, which is a yet another improved version of this dataset. Firstly, we identify and fix dialogue state annotation errors across 17.3% of the utterances on top of MultiWOZ 2.1. Secondly, we redefine the ontology by disallowing vocabularies of slots with a large number of possible values (e.g., restaurant name, time of booking). In addition, we introduce slot span annotations for these slots to standardize them across recent models, which previously used custom string matching heuristics to generate them. We also benchmark a few state of the art dialogue state tracking models on the corrected dataset to facilitate comparison for future work. In the end, we discuss best practices for dialogue data collection that can help avoid annotation errors.

Key facts. Cited by 201 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 51; 2023: 45; 2024: 36; 2025: 26; 2026: 15.

201 papers.

Sort:

By area:

  1. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  2. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  3. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 203
  4. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  5. 2023MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problemscited by 128
  6. 2021Dialogue State Tracking with a Language Model using Schema-Driven Promptingcited by 94
  7. 2021MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluationcited by 86
  8. 2023SpokenWOZ: A Large-Scale Speech-Text Benchmark for Spoken Task-Oriented Dialogue Agentscited by 71
  9. 2024Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmarkcited by 62
  10. 2021Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Trackingcited by 61
  11. 2021Zero-Shot Dialogue State Tracking via Cross-Task Transfercited by 52
  12. 2022Description-Driven Task-Oriented Dialog Modelingcited by 50
38 more
  1. 2022GODEL: Large-Scale Pre-Training for Goal-Directed Dialogcited by 50
  2. 2021Shades of BLEU, Flavours of Success: The Case of MultiWOZcited by 48
  3. 2022DeepStruct: Pretraining of Language Models for Structure Predictioncited by 45
  4. 2023SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Promptingcited by 39
  5. 2021PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generationcited by 37
  6. 2024SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Modelscited by 37
  7. 2022SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generationcited by 34
  8. 2024Large Language Models as Zero-shot Dialogue State Tracker through Function Callingcited by 34
  9. 2023Controllable Mixed-Initiative Dialogue Generation through Promptingcited by 32
  10. 2021Annotation Inconsistency and Entity Bias in MultiWOZcited by 30
  11. 2021"How Robust r u?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversationscited by 30
  12. 2021GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systemscited by 30
  13. 2022Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialogcited by 30
  14. 2023Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue Systemcited by 30
  15. 2022HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Datacited by 28
  16. 2024SPINACH: SPARQL-Based Information Navigation for Challenging Real-World Questionscited by 27
  17. 2025Navigating Rifts in Human-LLM Grounding: Study and Benchmarkcited by 27
  18. 2024Can Large Language Models Understand Context?cited by 26
  19. 2023Towards LLM-driven Dialogue State Trackingcited by 25
  20. 2024Enhancing Dialogue State Tracking Models through LLM-backed User-Agents Simulationcited by 24
  21. 2022Cross-Lingual Dialogue Dataset Creation via Outline-Based Generationcited by 23
  22. 2022"Do you follow me?": A Survey of Recent Approaches in Dialogue State Trackingcited by 22
  23. 2021Generalized Zero-shot Intent Detection via Commonsense Knowledgecited by 21
  24. 2021Amendable Generation for Dialogue State Trackingcited by 21
  25. 2022Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Trackingcited by 21
  26. 2022Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Datacited by 21
  27. 2023DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AIcited by 21
  28. 2021Effective Sequence-to-Sequence Dialogue State Trackingcited by 20
  29. 2022Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Trackingcited by 18
  30. 2024SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Trackingcited by 17
  31. 2021Learning to Select External Knowledge with Multi-Scale Negative Samplingcited by 16
  32. 2021Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systemscited by 16
  33. 2024Multi-News+: Cost-efficient Dataset Cleansing via LLM-based Data Annotationcited by 16
  34. 2022Robust Dialogue State Tracking with Weak Supervision and Sparse Datacited by 15
  35. 2022A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialogcited by 14
  36. 2023UMASS_BioNLP at MEDIQA-Chat 2023: Can LLMs generate high-quality synthetic note-oriented doctor-patient conversations?cited by 14
  37. 2022Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challengescited by 13
  38. 2022Towards Fair Evaluation of Dialogue State Tracking by Flexible Incorporation of Turn-level Performancescited by 13

50 most cited of 201 All 201 (JSON)