# Overview of the Ninth Dialog System Technology Challenge: DSTC9

arXiv: https://arxiv.org/abs/2011.06486 · HTML: https://mlsanity.com/paper/2011.06486/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2020.

## Built on by (most cited first, 30 of 49)

- [Evaluating Human-Language Model Interaction](https://mlsanity.com/paper/2212.09746/) (2022)
- [SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations](https://mlsanity.com/paper/2104.08667/) (2021)
- [GODEL: Large-Scale Pre-Training for Goal-Directed Dialog](https://mlsanity.com/paper/2206.11309/) (2022)
- [SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting](https://mlsanity.com/paper/2305.09067/) (2023)
- [ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?](https://mlsanity.com/paper/2306.01386/) (2023)
- [PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation](https://mlsanity.com/paper/2109.09519/) (2021)
- [AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models](https://mlsanity.com/paper/2102.05126/) (2021)
- [Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog](https://mlsanity.com/paper/2205.10400/) (2022)
- [X-Eval: Generalizable Multi-aspect Text Evaluation via Augmented Instruction Tuning with Auxiliary Evaluation Aspects](https://mlsanity.com/paper/2311.08788/) (2023)
- [DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations](https://mlsanity.com/paper/2203.09711/) (2022)
- ["Do you follow me?": A Survey of Recent Approaches in Dialogue State Tracking](https://mlsanity.com/paper/2207.14627/) (2022)
- [What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluation](https://mlsanity.com/paper/2203.13927/) (2022)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges](https://mlsanity.com/paper/2203.10012/) (2022)
- [FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation](https://mlsanity.com/paper/2210.13832/) (2022)
- [X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents](https://mlsanity.com/paper/2306.17674/) (2023)
- [Approximating Online Human Evaluation of Social Chatbots with Prompting](https://mlsanity.com/paper/2304.05253/) (2023)
- [Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues](https://mlsanity.com/paper/2111.02574/) (2021)
- [Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnostics](https://mlsanity.com/paper/2112.08321/) (2021)
- [Toward Self-learning End-to-End Task-Oriented Dialog Systems](https://mlsanity.com/paper/2201.06849/) (2022)
- [Interactive Evaluation of Dialog Track at DSTC9](https://mlsanity.com/paper/2207.14403/) (2022)
- [Towards Asking Clarification Questions for Information Seeking on Task-Oriented Dialogues](https://mlsanity.com/paper/2305.13690/) (2023)
- [State-of-the-art in Open-domain Conversational AI: A Survey](https://mlsanity.com/paper/2205.00965/) (2022)
- [Post-processing Networks: Method for Optimizing Pipeline Task-oriented Dialogue Systems using Reinforcement Learning](https://mlsanity.com/paper/2207.12185/) (2022)
- [Dialogue Term Extraction using Transfer Learning and Topological Data Analysis](https://mlsanity.com/paper/2208.10448/) (2022)
- [FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows](https://mlsanity.com/paper/2202.06633/) (2022)
- [Zero and Few-Shot Localization of Task-Oriented Dialogue Agents with a Distilled Representation](https://mlsanity.com/paper/2302.09424/) (2023)
- [Apollonion: Profile-centric Dialog Agent](https://mlsanity.com/paper/2404.08692/) (2024)
- [SYNERGY: Building Task Bots at Scale Using Symbolic Knowledge and Machine Teaching](https://mlsanity.com/paper/2110.11514/) (2021)
- [User Response and Sentiment Prediction for Automatic Dialogue Evaluation](https://mlsanity.com/paper/2111.08808/) (2021)

## Abstract (arXiv)

This paper introduces the Ninth Dialog System Technology Challenge (DSTC-9). This edition of the DSTC focuses on applying end-to-end dialog technologies for four distinct tasks in dialog systems, namely, 1. Task-oriented dialog Modeling with unstructured knowledge access, 2. Multi-domain task-oriented dialog, 3. Interactive evaluation of dialog, and 4. Situated interactive multi-modal dialog. This paper describes the task definition, provided datasets, baselines and evaluation set-up for each track. We also summarize the results of the submitted systems to highlight the overall trends of the state-of-the-art technologies for the tasks.