# CGoDial: A Large-Scale Benchmark for Chinese Goal-oriented Dialog Evaluation

arXiv: https://arxiv.org/abs/2211.11617 · HTML: https://mlsanity.com/paper/2211.11617/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 67 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 30 of 50)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [TEACh: Task-driven Embodied Agents that Chat](https://mlsanity.com/paper/2110.00534/) (2021)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking](https://mlsanity.com/paper/2005.02877/) (2020)
- [UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2](https://mlsanity.com/paper/2012.03539/) (2020)
- [StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding](https://mlsanity.com/paper/1908.04577/) (2019)
- [Learning End-to-End Goal-Oriented Dialog](https://mlsanity.com/paper/1605.07683/) (2016)
- [MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2009.12005/) (2020)
- [GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection](https://mlsanity.com/paper/2111.14592/) (2021)
- [Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context](https://mlsanity.com/paper/1911.10484/) (2019)
- [ConveRT: Efficient and Accurate Conversational Representations from Transformers](https://mlsanity.com/paper/1911.03688/) (2019)
- [DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue](https://mlsanity.com/paper/2009.13570/) (2020)
- [A Large-Scale Chinese Short-Text Conversation Dataset](https://mlsanity.com/paper/2008.03946/) (2020)
- [Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.00783/) (2021)
- [Building a Conversational Agent Overnight with Dialogue Self-Play](https://mlsanity.com/paper/1801.04871/) (2018)
- [CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset](https://mlsanity.com/paper/2002.11893/) (2020)
- [Modeling Multi-turn Conversation with Deep Utterance Aggregation](https://mlsanity.com/paper/1806.09102/) (2018)
- [Induction Networks for Few-Shot Text Classification](https://mlsanity.com/paper/1902.10482/) (2019)
- [CharBERT: Character-aware Pre-trained Language Model](https://mlsanity.com/paper/2011.01513/) (2020)
- [UER: An Open-Source Toolkit for Pre-training Models](https://mlsanity.com/paper/1909.05658/) (2019)
- [Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access](https://mlsanity.com/paper/2006.03533/) (2020)
- [SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations](https://mlsanity.com/paper/2104.08667/) (2021)
- [BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling](https://mlsanity.com/paper/2106.02787/) (2021)
- [Training Neural Response Selection for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1906.01543/) (2019)
- [A Repository of Conversational Datasets](https://mlsanity.com/paper/1904.06472/) (2019)

## Built on by (most cited first, 9 of 9)

- [SpokenWOZ: A Large-Scale Speech-Text Benchmark for Spoken Task-Oriented Dialogue Agents](https://mlsanity.com/paper/2305.13040/) (2023)
- [MemGuide: Intent-Driven Memory Selection for Goal-Oriented Multi-Session LLM Agents](https://mlsanity.com/paper/2505.20231/) (2025)
- [Speech-Text Dialog Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment](https://mlsanity.com/paper/2305.11579/) (2023)
- [U-NEED: A Fine-grained Dataset for User Needs-Centric E-commerce Conversational Recommendation](https://mlsanity.com/paper/2305.04774/) (2023)
- [Unsupervised Dialogue Topic Segmentation with Topic-aware Utterance Representation](https://mlsanity.com/paper/2305.02747/) (2023)
- [Empathetic Response Generation via Emotion Cause Transition Graph](https://mlsanity.com/paper/2302.11787/) (2023)
- [Coarse-to-Fine Knowledge Selection for Document Grounded Dialogs](https://mlsanity.com/paper/2302.11849/) (2023)
- [JMultiWOZ: A Large-Scale Japanese Multi-Domain Task-Oriented Dialogue Dataset](https://mlsanity.com/paper/2403.17319/) (2024)
- [DiagFlowBench: Evaluating How Language Models Handle Off-Procedure Inputs in Grounded Diagnostic Dialogue](https://mlsanity.com/paper/2606.17904/) (2026)

## Abstract (arXiv)

Practical dialog systems need to deal with various knowledge sources, noisy user expressions, and the shortage of annotated data. To better solve the above problems, we propose CGoDial, new challenging and comprehensive Chinese benchmark for multi-domain Goal-oriented Dialog evaluation. It contains 96,763 dialog sessions and 574,949 dialog turns totally, covering three datasets with different knowledge sources: 1) a slot-based dialog (SBD) dataset with table-formed knowledge, 2) a flow-based dialog (FBD) dataset with tree-formed knowledge, and a retrieval-based dialog (RBD) dataset with candidate-formed knowledge. To bridge the gap between academic benchmarks and spoken dialog scenarios, we either collect data from real conversations or add spoken features to existing datasets via crowd-sourcing. The proposed experimental settings include the combinations of training with either the entire training set or a few-shot training set, and testing with either the standard test set or a hard test subset, which can assess model capabilities in terms of general prediction, fast adaptability and reliable robustness.