# A Simple but Effective BERT Model for Dialog State Tracking on Resource-Limited Systems

arXiv: https://arxiv.org/abs/1910.12995 · HTML: https://mlsanity.com/paper/1910.12995/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2019.

## Built on by (most cited first, 10 of 10)

- [ReMeDi: Resources for Multi-domain, Multi-service, Medical Dialogues](https://mlsanity.com/paper/2109.00430/) (2021)
- [End-to-end Neural Coreference Resolution Revisited: A Simple yet Effective Baseline](https://mlsanity.com/paper/2107.01700/) (2021)
- [ASR-Generated Text for Language Model Pre-training Applied to Speech Tasks](https://mlsanity.com/paper/2207.01893/) (2022)
- [OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue](https://mlsanity.com/paper/2209.04595/) (2022)
- [Prompt Backdoors in Visual Prompt Learning](https://mlsanity.com/paper/2310.07632/) (2023)
- [XQA-DST: Multi-Domain and Multi-Lingual Dialogue State Tracking](https://mlsanity.com/paper/2204.05895/) (2022)
- [Thread Detection and Response Generation using Transformers with Prompt Optimisation](https://mlsanity.com/paper/2403.05931/) (2024)
- [LMN at SemEval-2022 Task 11: A Transformer-based System for English Named Entity Recognition](https://mlsanity.com/paper/2203.03546/) (2022)
- [Ensemble Transfer Learning for Multilingual Coreference Resolution](https://mlsanity.com/paper/2301.09175/) (2023)
- [Injecting linguistic knowledge into BERT for Dialogue State Tracking](https://mlsanity.com/paper/2311.15623/) (2023)

## Abstract (arXiv)

In a task-oriented dialog system, the goal of dialog state tracking (DST) is to monitor the state of the conversation from the dialog history. Recently, many deep learning based methods have been proposed for the task. Despite their impressive performance, current neural architectures for DST are typically heavily-engineered and conceptually complex, making it difficult to implement, debug, and maintain them in a production setting. In this work, we propose a simple but effective DST model based on BERT. In addition to its simplicity, our approach also has a number of other advantages: (a) the number of parameters does not grow with the ontology size (b) the model can operate in situations where the domain ontology may change dynamically. Experimental results demonstrate that our BERT-based model outperforms previous methods by a large margin, achieving new state-of-the-art results on the standard WoZ 2.0 dataset. Finally, to make the model small and fast enough for resource-restricted systems, we apply the knowledge distillation method to compress our model. The final compressed model achieves comparable results with the original model while being 8x smaller and 7x faster.