mlsanityData to 7 Oct 2026

Overview of the Ninth Dialog System Technology Challenge: DSTC9

arXiv 2011.06486 ↗First posted 12 Nov 2020cs.CLcited by 49

Not on the map. Why?

Abstract (arXiv)

This paper introduces the Ninth Dialog System Technology Challenge (DSTC-9). This edition of the DSTC focuses on applying end-to-end dialog technologies for four distinct tasks in dialog systems, namely, 1. Task-oriented dialog Modeling with unstructured knowledge access, 2. Multi-domain task-oriented dialog, 3. Interactive evaluation of dialog, and 4. Situated interactive multi-modal dialog. This paper describes the task definition, provided datasets, baselines and evaluation set-up for each track. We also summarize the results of the submitted systems to highlight the overall trends of the state-of-the-art technologies for the tasks.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 12; 2022: 23; 2023: 11; 2024: 3.

49 papers.

Sort:

By area:

  1. 2022Evaluating Human-Language Model Interactioncited by 90
  2. 2021SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversationscited by 54
  3. 2022GODEL: Large-Scale Pre-Training for Goal-Directed Dialogcited by 50
  4. 2023SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Promptingcited by 39
  5. 2023ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?cited by 38
  6. 2021PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generationcited by 37
  7. 2021AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Modelscited by 31
  8. 2022Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialogcited by 30
  9. 2023X-Eval: Generalizable Multi-aspect Text Evaluation via Augmented Instruction Tuning with Auxiliary Evaluation Aspectscited by 28
  10. 2022DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulationscited by 25
  11. 2022"Do you follow me?": A Survey of Recent Approaches in Dialogue State Trackingcited by 22
  12. 2022What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluationcited by 18
37 more
  1. 2021Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systemscited by 16
  2. 2022Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challengescited by 13
  3. 2022FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluationcited by 13
  4. 2023X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agentscited by 12
  5. 2023Approximating Online Human Evaluation of Social Chatbots with Promptingcited by 11
  6. 2021Contextual Semantic Parsing for Multilingual Task-Oriented Dialoguescited by 10
  7. 2021Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnosticscited by 9
  8. 2022Toward Self-learning End-to-End Task-Oriented Dialog Systemscited by 9
  9. 2022Interactive Evaluation of Dialog Track at DSTC9cited by 9
  10. 2023Towards Asking Clarification Questions for Information Seeking on Task-Oriented Dialoguescited by 7
  11. 2022State-of-the-art in Open-domain Conversational AI: A Surveycited by 5
  12. 2022Post-processing Networks: Method for Optimizing Pipeline Task-oriented Dialogue Systems using Reinforcement Learningcited by 5
  13. 2022Dialogue Term Extraction using Transfer Learning and Topological Data Analysiscited by 5
  14. 2022FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flowscited by 4
  15. 2023Zero and Few-Shot Localization of Task-Oriented Dialogue Agents with a Distilled Representationcited by 4
  16. 2024Apollonion: Profile-centric Dialog Agentcited by 4
  17. 2021SYNERGY: Building Task Bots at Scale Using Symbolic Knowledge and Machine Teachingcited by 3
  18. 2021User Response and Sentiment Prediction for Automatic Dialogue Evaluationcited by 3
  19. 2022Dialogue Evaluation with Offline Reinforcement Learningcited by 3
  20. 2022Topic-Aware Response Generation in Task-Oriented Dialogue with Unstructured Knowledge Accesscited by 3
  21. 2022PoE: a Panel of Experts for Generalized Automatic Dialogue Assessmentcited by 3
  22. 2022EVI: Multilingual Spoken Dialogue Tasks and Dataset for Knowledge-Based Enrolment, Verification, and Identificationcited by 2
  23. 2022CookDial: A dataset for task-oriented dialogs grounded in procedural documentscited by 2
  24. 2022SelF-Eval: Self-supervised Fine-grained Dialogue Evaluationcited by 2
  25. 2022Evaluating Agent Interactions Through Episodic Knowledge Graphscited by 2
  26. 2023Open-Domain Text Evaluation via Contrastive Distribution Methodscited by 1
  27. 2023Evaluating Chatbots to Promote Users' Trust -- Practices and Open Problemscited by 1
  28. 2021WeChat AI & ICT's Submission for DSTC9 Interactive Dialogue Evaluation Trackcited by 0
  29. 2021Did Chatbots Miss Their 'Apollo Moment'? A Survey of the Potential, Gaps and Lessons from Using Collaboration Assistants During COVID-19cited by 0
  30. 2021Efficient Dialogue State Tracking by Masked Hierarchical Transformercited by 0
  31. 2021Investigating Effect of Dialogue History in Multilingual Task Oriented Dialogue Systemscited by 0
  32. 2022ViWOZ: A Multi-Domain Task-Oriented Dialogue Systems Dataset For Low-resource Languagecited by 0
  33. 2022The DialPort toolscited by 0
  34. 2023Multimodal Recommendation Dialog with Subjective Preference: A New Challenge and Benchmarkcited by 0
  35. 2023Athena 2.0: Discourse and User Modeling in Open Domain Dialoguecited by 0
  36. 2024Online vs Offline: A Comparative Study of First-Party and Third-Party Evaluations of Social Chatbotscited by 0
  37. 2024Building Dialogue Understanding Models for Low-resource Language Indonesian from Scratchcited by 0