mlsanityData to 7 Oct 2026

MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluation

arXiv 2104.00773 ↗First posted 1 Apr 2021cs.CLcited by 86

Not on the map. Why?

Abstract (arXiv)

The MultiWOZ 2.0 dataset has greatly stimulated the research of task-oriented dialogue systems. However, its state annotations contain substantial noise, which hinders a proper evaluation of model performance. To address this issue, massive efforts were devoted to correcting the annotations. Three improved versions (i.e., MultiWOZ 2.1-2.3) have then been released. Nonetheless, there are still plenty of incorrect and inconsistent annotations. This work introduces MultiWOZ 2.4, which refines the annotations in the validation set and test set of MultiWOZ 2.1. The annotations in the training set remain unchanged (same as MultiWOZ 2.1) to elicit robust and noise-resilient model training. We benchmark eight state-of-the-art dialogue state tracking models on MultiWOZ 2.4. All of them demonstrate much higher performance than on MultiWOZ 2.1.

Key facts. Cited by 86 papers in the map (data through 7 Oct 2026). Builds on 24 of its 33 references that are in the map. First posted on arXiv 1 Apr 2021.

Builds on

24 of 33 references.

References · 24

  1. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  2. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  3. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  4. 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 426
  5. 2020A Simple Language Model for Task-Oriented Dialoguecited by 331
  6. 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
  7. 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 215
  8. 2020MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselinescited by 201
  9. 2016Neural Belief Tracker: Data-Driven Dialogue State Trackingcited by 142
  10. 2020TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Trackingcited by 130
  11. 2020A Survey of Label-noise Representation Learning: Past, Present and Futurecited by 121
  12. 2017Frames: A Corpus for Adding Memory to Goal-Oriented Dialogue Systemscited by 84
12 more
  1. 2019Efficient Dialogue State Tracking by Selectively Overwriting Memorycited by 76
  2. 2019Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Trackingcited by 75
  3. 2020CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Datasetcited by 66
  4. 2019SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Trackingcited by 63
  5. 2020MultiWOZ 2.3: A multi-domain task-oriented dialogue dataset enhanced with annotation corrections and co-reference annotationcited by 51
  6. 2018Global-Locally Self-Attentive Dialogue State Trackercited by 45
  7. 2020CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackerscited by 43
  8. 2020RiSAWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset with Rich Semantic Annotations for Task-Oriented Dialogue Modelingcited by 40
  9. 2020Conversational Semantic Parsing for Dialog State Trackingcited by 40
  10. 2021Slot Self-Attentive Dialogue State Trackingcited by 27
  11. 2020A Contextual Hierarchical Attention Network with Adaptive Objective for Dialogue State Trackingcited by 20
  12. 2020Parallel Interactive Networks for Multi-Domain Dialogue State Generationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 18; 2023: 16; 2024: 18; 2025: 15; 2026: 12.

86 papers.

Sort:

By area:

  1. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  2. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  3. 2023MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problemscited by 128
  4. 2022Description-Driven Task-Oriented Dialog Modelingcited by 50
  5. 2022In-Context Learning for Few-Shot Dialogue State Trackingcited by 47
  6. 2024API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMscited by 39
  7. 2023A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understandingcited by 31
  8. 2021Fusing task-oriented and open-domain dialogues in conversational agentscited by 28
  9. 2023Towards LLM-driven Dialogue State Trackingcited by 25
  10. 2024Enhancing Dialogue State Tracking Models through LLM-backed User-Agents Simulationcited by 24
  11. 2022"Do you follow me?": A Survey of Recent Approaches in Dialogue State Trackingcited by 22
  12. 2022Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Trackingcited by 21
38 more
  1. 2021Effective Sequence-to-Sequence Dialogue State Trackingcited by 20
  2. 2025Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Modelcited by 20
  3. 2025Goal Alignment in LLM-Based User Simulators for Conversational AIcited by 19
  4. 2023OrchestraLLM: Efficient Orchestration of Language Models for Dialogue State Trackingcited by 17
  5. 2024Multi-News+: Cost-efficient Dataset Cleansing via LLM-based Data Annotationcited by 16
  6. 2022Robust Dialogue State Tracking with Weak Supervision and Sparse Datacited by 15
  7. 2022A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialogcited by 14
  8. 2023Unraveling ChatGPT: A Critical Analysis of AI-Generated Goal-Oriented Dialogues and Annotationscited by 14
  9. 2026A Survey on LLM-based Conversational User Simulationcited by 14
  10. 2025Enhancing LLM Reliability via Explicit Knowledge Boundary Modelingcited by 13
  11. 2022Workflow Discovery from Dialogues in the Low Data Regimecited by 12
  12. 2025MIRAGE: A Benchmark for Multimodal Information-Seeking and Reasoning in Agricultural Expert-Guided Conversationscited by 12
  13. 2022ASSIST: Towards Label Noise-Robust Dialogue State Trackingcited by 11
  14. 2021Contextual Semantic Parsing for Multilingual Task-Oriented Dialoguescited by 10
  15. 2023Diverse Retrieval-Augmented In-Context Learning for Dialogue State Trackingcited by 10
  16. 2025TD-EVAL: Revisiting Task-Oriented Dialogue Evaluation by Combining Turn-Level Precision with Dialogue-Level Comparisonscited by 10
  17. 2025Frictional Agent Alignment Framework: Slow Down and Don't Break Thingscited by 10
  18. 2021AllWOZ: Towards Multilingual Task-Oriented Dialog Systems for Allcited by 9
  19. 2021Database Search Results Disambiguation for Task-Oriented Dialog Systemscited by 9
  20. 2024DARD: A Multi-Agent Approach for Task-Oriented Dialog Systemscited by 9
  21. 2024Effective and Efficient Conversation Retrieval for Dialogue State Tracking with Implicit Text Summariescited by 8
  22. 2024Continual Dialogue State Tracking via Reason-of-Select Distillationcited by 8
  23. 2023S3-DST: Structured Open-Domain Dialogue Segmentation and State Tracking in the Era of LLMscited by 7
  24. 2022Injecting Domain Knowledge in Language Models for Task-Oriented Dialogue Systemscited by 6
  25. 2024Diverse and Effective Synthetic Data Generation for Adaptable Zero-Shot Dialogue State Trackingcited by 6
  26. 2022A Multi-Task BERT Model for Schema-Guided Dialogue State Trackingcited by 5
  27. 2022Knowledge-grounded Dialog State Trackingcited by 5
  28. 2023Semantic Parsing by Large Language Models for Intricate Updating Strategies of Zero-Shot Dialogue State Trackingcited by 5
  29. 2023Diable: Efficient Dialogue State Tracking as Operations on Tablescited by 4
  30. 2022MetaASSIST: Robust Dialogue State Tracking with Meta Learningcited by 3
  31. 2022MoNET: Tackle State Momentum via Noise-Enhanced Training for Dialogue State Trackingcited by 3
  32. 2026TurnWise: The Gap between Single- and Multi-turn Language Model Capabilitiescited by 3
  33. 2021Oh My Mistake!: Toward Realistic Dialogue State Tracking including Turnback Utterancescited by 2
  34. 2021POSSCORE: A Simple Yet Effective Evaluation of Conversational Search with Part of Speech Labellingcited by 2
  35. 2022What Did You Say? Task-Oriented Dialog Datasets Are Not Conversational!?cited by 2
  36. 2023Span-Selective Linear Attention Transformers for Effective and Robust Schema-Guided Dialogue State Trackingcited by 2
  37. 2023Unified Conversational Models with System-Initiated Transitions between Chit-Chat and Task-Oriented Dialoguescited by 2
  38. 2023System-Initiated Transitions from Chit-Chat to Task-Oriented Dialogues with Transition Info Extractor and Transition Sentence Generatorcited by 2

50 most cited of 86 All 86 (JSON)