# Zero-Shot Dialogue State Tracking via Cross-Task Transfer

arXiv: https://arxiv.org/abs/2109.04655 · HTML: https://mlsanity.com/paper/2109.04655/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 51 references that are in the map.
- First posted on arXiv 10 Sep 2021.

## Builds on (most cited first, 30 of 42)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine](https://mlsanity.com/paper/1704.05179/) (2017)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines](https://mlsanity.com/paper/2007.12720/) (2020)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension](https://mlsanity.com/paper/1910.09753/) (2019)
- [DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension](https://mlsanity.com/paper/1902.00164/) (2019)
- [TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking](https://mlsanity.com/paper/2005.02877/) (2020)
- [MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2009.12005/) (2020)
- [Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context](https://mlsanity.com/paper/1911.10484/) (2019)
- [DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue](https://mlsanity.com/paper/2009.13570/) (2020)
- [Continual Learning in Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2012.15504/) (2020)
- [Efficient Dialogue State Tracking by Selectively Overwriting Memory](https://mlsanity.com/paper/1911.03906/) (2019)

## Built on by (most cited first, 30 of 52)

- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Few-Shot Bot: Prompt-Based Learning for Dialogue Systems](https://mlsanity.com/paper/2110.08118/) (2021)
- [Description-Driven Task-Oriented Dialog Modeling](https://mlsanity.com/paper/2201.08904/) (2022)
- [Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data](https://mlsanity.com/paper/2301.05843/) (2023)
- [In-Context Learning for Few-Shot Dialogue State Tracking](https://mlsanity.com/paper/2203.08568/) (2022)
- [SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting](https://mlsanity.com/paper/2305.09067/) (2023)
- [ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?](https://mlsanity.com/paper/2306.01386/) (2023)
- [Large Language Models as Zero-shot Dialogue State Tracker through Function Calling](https://mlsanity.com/paper/2402.10466/) (2024)
- [Towards LLM-driven Dialogue State Tracking](https://mlsanity.com/paper/2310.14970/) (2023)
- ["Do you follow me?": A Survey of Recent Approaches in Dialogue State Tracking](https://mlsanity.com/paper/2207.14627/) (2022)
- [Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking](https://mlsanity.com/paper/2203.01552/) (2022)
- [Show, Don't Tell: Demonstrations Outperform Descriptions for Schema-Guided Task-Oriented Dialogue](https://mlsanity.com/paper/2204.04327/) (2022)
- [SalesBot: Transitioning from Chit-Chat to Task-Oriented Dialogues](https://mlsanity.com/paper/2204.10591/) (2022)
- [Robust Dialogue State Tracking with Weak Supervision and Sparse Data](https://mlsanity.com/paper/2202.03354/) (2022)
- [CHATEDIT: Towards Multi-turn Interactive Facial Image Editing via Dialogue](https://mlsanity.com/paper/2303.11108/) (2023)
- [Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking](https://mlsanity.com/paper/2306.00434/) (2023)
- [Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking](https://mlsanity.com/paper/2307.01453/) (2023)
- [DiSTRICT: Dialogue State Tracking with Retriever Driven In-Context Tuning](https://mlsanity.com/paper/2212.02851/) (2022)
- [A Dual Prompt Learning Framework for Few-Shot Dialogue State Tracking](https://mlsanity.com/paper/2201.05780/) (2022)
- [Parameter-Efficient Low-Resource Dialogue State Tracking by Prompt Tuning](https://mlsanity.com/paper/2301.10915/) (2023)
- [Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation](https://mlsanity.com/paper/2306.04724/) (2023)
- [Teaching Models new APIs: Domain-Agnostic Simulators for Task Oriented Dialogue](https://mlsanity.com/paper/2110.06905/) (2021)
- [ZEROTOP: Zero-Shot Task-Oriented Semantic Parsing using Large Language Models](https://mlsanity.com/paper/2212.10815/) (2022)
- [InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2310.08885/) (2023)
- [Semantic Parsing by Large Language Models for Intricate Updating Strategies of Zero-Shot Dialogue State Tracking](https://mlsanity.com/paper/2310.10520/) (2023)
- [Continual Dialogue State Tracking via Example-Guided Question Answering](https://mlsanity.com/paper/2305.13721/) (2023)
- [Confidence Estimation for LLM-Based Dialogue State Tracking](https://mlsanity.com/paper/2409.09629/) (2024)
- [Zero-Shot Generalizable End-to-End Task-Oriented Dialog System using Context Summarization and Domain Schema](https://mlsanity.com/paper/2303.16252/) (2023)
- [SQATIN: Supervised Instruction Tuning Meets Question Answering for Improved Dialogue NLU](https://mlsanity.com/paper/2311.09502/) (2023)

## Abstract (arXiv)

Zero-shot transfer learning for dialogue state tracking (DST) enables us to handle a variety of task-oriented dialogue domains without the expense of collecting in-domain data. In this work, we propose to transfer the textitcross-task knowledge from general question answering (QA) corpora for the zero-shot DST task. Specifically, we propose TransferQA, a transferable generative QA model that seamlessly combines extractive QA and multi-choice QA via a text-to-text transformer framework, and tracks both categorical slots and non-categorical slots in DST. In addition, we introduce two effective ways to construct unanswerable questions, namely, negative question sampling and context truncation, which enable our model to handle "none" value slots in the zero-shot DST setting. The extensive experiments show that our approaches substantially improve the existing zero-shot and few-shot results on MultiWoz. Moreover, compared to the fully trained baseline on the Schema-Guided Dialogue dataset, our approach shows better generalization ability in unseen domains.