# Fine-Tuning BERT for Schema-Guided Zero-Shot Dialogue State Tracking

arXiv: https://arxiv.org/abs/2002.00181 · HTML: https://mlsanity.com/paper/2002.00181/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2020.

## Built on by (most cited first, 11 of 11)

- [Description-Driven Task-Oriented Dialog Modeling](https://mlsanity.com/paper/2201.08904/) (2022)
- [SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems](https://mlsanity.com/paper/2110.06800/) (2021)
- [MemGuide: Intent-Driven Memory Selection for Goal-Oriented Multi-Session LLM Agents](https://mlsanity.com/paper/2505.20231/) (2025)
- [Show, Don't Tell: Demonstrations Outperform Descriptions for Schema-Guided Task-Oriented Dialogue](https://mlsanity.com/paper/2204.04327/) (2022)
- [A Multi-Task BERT Model for Schema-Guided Dialogue State Tracking](https://mlsanity.com/paper/2207.00828/) (2022)
- [SGD-QA: Fast Schema-Guided Dialogue State Tracking for Unseen Services](https://mlsanity.com/paper/2105.08049/) (2021)
- [What Did You Say? Task-Oriented Dialog Datasets Are Not Conversational!?](https://mlsanity.com/paper/2203.03431/) (2022)
- [Span-Selective Linear Attention Transformers for Effective and Robust Schema-Guided Dialogue State Tracking](https://mlsanity.com/paper/2306.09340/) (2023)
- [Choice Fusion as Knowledge for Zero-Shot Dialogue State Tracking](https://mlsanity.com/paper/2302.13013/) (2023)
- [Multi-Stage Coarse-to-Fine Contrastive Learning for Conversation Intent Induction](https://mlsanity.com/paper/2303.05034/) (2023)
- [HR-Agent: A Task-Oriented Dialogue (TOD) LLM Agent Tailored for HR Applications](https://mlsanity.com/paper/2410.11239/) (2024)

## Abstract (arXiv)

We present our work on Track 4 in the Dialogue System Technology Challenges 8 (DSTC8). The DSTC8-Track 4 aims to perform dialogue state tracking (DST) under the zero-shot settings, in which the model needs to generalize on unseen service APIs given a schema definition of these target APIs. Serving as the core for many virtual assistants such as Siri, Alexa, and Google Assistant, the DST keeps track of the user's goal and what happened in the dialogue history, mainly including intent prediction, slot filling, and user state tracking, which tests models' ability of natural language understanding. Recently, the pretrained language models have achieved state-of-the-art results and shown impressive generalization ability on various NLP tasks, which provide a promising way to perform zero-shot learning for language understanding. Based on this, we propose a schema-guided paradigm for zero-shot dialogue state tracking (SGP-DST) by fine-tuning BERT, one of the most popular pretrained language models. The SGP-DST system contains four modules for intent prediction, slot prediction, slot transfer prediction, and user state summarizing respectively. According to the official evaluation results, our SGP-DST (team12) ranked 3rd on the joint goal accuracy (primary evaluation metric for ranking submissions) and 1st on the requsted slots F1 among 25 participant teams.