# Dialogue State Distillation Network with Inter-slot Contrastive Learning for Dialogue State Tracking

arXiv: https://arxiv.org/abs/2302.08220 · HTML: https://mlsanity.com/paper/2302.08220/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 46 references that are in the map.
- First posted on arXiv 16 Feb 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System](https://mlsanity.com/paper/2109.14739/) (2021)
- [TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking](https://mlsanity.com/paper/2005.02877/) (2020)
- [Recent Advances and Challenges in Task-oriented Dialog System](https://mlsanity.com/paper/2003.07490/) (2020)
- [Efficient Dialogue State Tracking by Selectively Overwriting Memory](https://mlsanity.com/paper/1911.03906/) (2019)
- [Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking](https://mlsanity.com/paper/1910.03544/) (2019)
- [CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning](https://mlsanity.com/paper/2109.07589/) (2021)
- [SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking](https://mlsanity.com/paper/1907.07421/) (2019)
- [Dialog State Tracking: A Neural Reading Comprehension Approach](https://mlsanity.com/paper/1908.01946/) (2019)
- [BERT Learns to Teach: Knowledge Distillation with Meta Learning](https://mlsanity.com/paper/2106.04570/) (2021)
- [PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks](https://mlsanity.com/paper/2202.12499/) (2022)
- [CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding](https://mlsanity.com/paper/2107.00440/) (2021)
- [Global-Locally Self-Attentive Dialogue State Tracker](https://mlsanity.com/paper/1805.09655/) (2018)
- [CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackers](https://mlsanity.com/paper/2010.12850/) (2020)
- [Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialog State Tracking](https://mlsanity.com/paper/2106.00291/) (2021)
- [Towards Universal Dialogue State Tracking](https://mlsanity.com/paper/1810.09587/) (2018)
- [CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems](https://mlsanity.com/paper/2109.04645/) (2021)
- [Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing](https://mlsanity.com/paper/1807.06517/) (2018)
- [An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking](https://mlsanity.com/paper/1805.01555/) (2018)
- [Efficient Context and Schema Fusion Networks for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2004.03386/) (2020)
- [Slot Self-Attentive Dialogue State Tracking](https://mlsanity.com/paper/2101.09374/) (2021)
- [Knowledge-Aware Graph-Enhanced GPT-2 for Dialogue State Tracking](https://mlsanity.com/paper/2104.04466/) (2021)
- [Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2004.10171/) (2020)
- [Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2204.06677/) (2022)

## Built on by (most cited first, 1 of 1)

- [From Words to Wisdom: Discourse Annotation and Baseline Models for Student Dialogue Understanding](https://mlsanity.com/paper/2511.20547/) (2025)

## Abstract (arXiv)

In task-oriented dialogue systems, Dialogue State Tracking (DST) aims to extract users' intentions from the dialogue history. Currently, most existing approaches suffer from error propagation and are unable to dynamically select relevant information when utilizing previous dialogue states. Moreover, the relations between the updates of different slots provide vital clues for DST. However, the existing approaches rely only on predefined graphs to indirectly capture the relations. In this paper, we propose a Dialogue State Distillation Network (DSDN) to utilize relevant information of previous dialogue states and migrate the gap of utilization between training and testing. Thus, it can dynamically exploit previous dialogue states and avoid introducing error propagation simultaneously. Further, we propose an inter-slot contrastive learning loss to effectively capture the slot co-update relations from dialogue context. Experiments are conducted on the widely used MultiWOZ 2.0 and MultiWOZ 2.1 datasets. The experimental results show that our proposed model achieves the state-of-the-art performance for DST.