# BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformer

arXiv: https://arxiv.org/abs/1907.03040 · HTML: https://mlsanity.com/paper/1907.03040/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2019.

## Built on by (most cited first, 30 of 38)

- [Dialogue State Tracking with a Language Model using Schema-Driven Prompting](https://mlsanity.com/paper/2109.07506/) (2021)
- [SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations](https://mlsanity.com/paper/2104.08667/) (2021)
- [In-Context Learning for Few-Shot Dialogue State Tracking](https://mlsanity.com/paper/2203.08568/) (2022)
- [CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems](https://mlsanity.com/paper/2109.04645/) (2021)
- [A Short Survey of Pre-trained Language Models for Conversational AI-A NewAge in NLP](https://mlsanity.com/paper/2104.10810/) (2021)
- [Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation](https://mlsanity.com/paper/2201.13405/) (2022)
- [Amendable Generation for Dialogue State Tracking](https://mlsanity.com/paper/2110.15659/) (2021)
- [ReMeDi: Resources for Multi-domain, Multi-service, Medical Dialogues](https://mlsanity.com/paper/2109.00430/) (2021)
- [MemGuide: Intent-Driven Memory Selection for Goal-Oriented Multi-Session LLM Agents](https://mlsanity.com/paper/2505.20231/) (2025)
- [Robust Dialogue State Tracking with Weak Supervision and Sparse Data](https://mlsanity.com/paper/2202.03354/) (2022)
- [Post-Training Large Language Models via Reinforcement Learning from Self-Feedback](https://mlsanity.com/paper/2507.21931/) (2025)
- [Improved and Efficient Conversational Slot Labeling through Question Answering](https://mlsanity.com/paper/2204.02123/) (2022)
- [Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking](https://mlsanity.com/paper/2306.00434/) (2023)
- [Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performance](https://mlsanity.com/paper/2109.04349/) (2021)
- [Beyond the Granularity: Multi-Perspective Dialogue Collaborative Selection for Dialogue State Tracking](https://mlsanity.com/paper/2205.10059/) (2022)
- [Comprehensive Study: How the Context Information of Different Granularity Affects Dialogue State Tracking?](https://mlsanity.com/paper/2105.03571/) (2021)
- [On Tracking Dialogue State by Inheriting Slot Values in Mentioned Slot Pools](https://mlsanity.com/paper/2202.07156/) (2022)
- [Zero-Shot Cross-Domain Dialogue State Tracking via Dual Low-Rank Adaptation](https://mlsanity.com/paper/2407.21633/) (2024)
- [Mismatch between Multi-turn Dialogue and its Evaluation Metric in Dialogue State Tracking](https://mlsanity.com/paper/2203.03123/) (2022)
- [Diable: Efficient Dialogue State Tracking as Operations on Tables](https://mlsanity.com/paper/2305.17020/) (2023)
- [Task Conditioned BERT for Joint Intent Detection and Slot-filling](https://mlsanity.com/paper/2308.06165/) (2023)
- [Improving Longer-range Dialogue State Tracking](https://mlsanity.com/paper/2103.00109/) (2021)
- [XQA-DST: Multi-Domain and Multi-Lingual Dialogue State Tracking](https://mlsanity.com/paper/2204.05895/) (2022)
- [SF-DST: Few-Shot Self-Feeding Reading Comprehension Dialogue State Tracking with Auxiliary Task](https://mlsanity.com/paper/2209.07742/) (2022)
- [Span-Selective Linear Attention Transformers for Effective and Robust Schema-Guided Dialogue State Tracking](https://mlsanity.com/paper/2306.09340/) (2023)
- [Thread Detection and Response Generation using Transformers with Prompt Optimisation](https://mlsanity.com/paper/2403.05931/) (2024)
- [ShopTalk: A System for Conversational Faceted Search](https://mlsanity.com/paper/2109.00702/) (2021)
- [Multi-Task Learning for Situated Multi-Domain End-to-End Dialogue Systems](https://mlsanity.com/paper/2110.05221/) (2021)
- [An Empirical Study of Multitask Learning to Improve Open Domain Dialogue Systems](https://mlsanity.com/paper/2304.08115/) (2023)
- [Exploring the Viability of Synthetic Audio Data for Audio-Based Dialogue State Tracking](https://mlsanity.com/paper/2312.01842/) (2023)

## Abstract (arXiv)

An important yet rarely tackled problem in dialogue state tracking (DST) is scalability for dynamic ontology (e.g., movie, restaurant) and unseen slot values. We focus on a specific condition, where the ontology is unknown to the state tracker, but the target slot value (except for none and dontcare), possibly unseen during training, can be found as word segment in the dialogue context. Prior approaches often rely on candidate generation from n-gram enumeration or slot tagger outputs, which can be inefficient or suffer from error propagation. We propose BERT-DST, an end-to-end dialogue state tracker which directly extracts slot values from the dialogue context. We use BERT as dialogue context encoder whose contextualized language representations are suitable for scalable DST to identify slot values from their semantic context. Furthermore, we employ encoder parameter sharing across all slots with two advantages: (1) Number of parameters does not grow linearly with the ontology. (2) Language representation knowledge can be transferred among slots. Empirical evaluation shows BERT-DST with cross-slot parameter sharing outperforms prior work on the benchmark scalable DST datasets Sim-M and Sim-R, and achieves competitive performance on the standard DSTC2 and WOZ 2.0 datasets.