# Slot Self-Attentive Dialogue State Tracking

arXiv: https://arxiv.org/abs/2101.09374 · HTML: https://mlsanity.com/paper/2101.09374/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 65 references that are in the map.
- First posted on arXiv 22 Jan 2021.

## Builds on (most cited first, 30 of 43)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [A Survey on Dialogue Systems: Recent Advances and New Frontiers](https://mlsanity.com/paper/1711.01731/) (2017)
- [Neural Belief Tracker: Data-Driven Dialogue State Tracking](https://mlsanity.com/paper/1606.03777/) (2016)
- [TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking](https://mlsanity.com/paper/2005.02877/) (2020)
- [MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2009.12005/) (2020)
- [Efficient Dialogue State Tracking by Selectively Overwriting Memory](https://mlsanity.com/paper/1911.03906/) (2019)
- [Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking](https://mlsanity.com/paper/1910.03544/) (2019)
- [SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking](https://mlsanity.com/paper/1907.07421/) (2019)
- [Dialog State Tracking: A Neural Reading Comprehension Approach](https://mlsanity.com/paper/1908.01946/) (2019)
- [Global-Locally Self-Attentive Dialogue State Tracker](https://mlsanity.com/paper/1805.09655/) (2018)
- [From Machine Reading Comprehension to Dialogue State Tracking: Bridging the Gap](https://mlsanity.com/paper/2004.05827/) (2020)
- [Towards Universal Dialogue State Tracking](https://mlsanity.com/paper/1810.09587/) (2018)
- [An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking](https://mlsanity.com/paper/1805.01555/) (2018)
- [Efficient Context and Schema Fusion Networks for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2004.03386/) (2020)
- [Scalable Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/1712.10224/) (2017)
- [Scalable and Accurate Dialogue State Tracking via Hierarchical Sequence Generation](https://mlsanity.com/paper/1909.00754/) (2019)

## Built on by (most cited first, 27 of 27)

- [Dialogue State Tracking with a Language Model using Schema-Driven Prompting](https://mlsanity.com/paper/2109.07506/) (2021)
- [MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluation](https://mlsanity.com/paper/2104.00773/) (2021)
- [Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Tracking](https://mlsanity.com/paper/2105.04222/) (2021)
- [ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?](https://mlsanity.com/paper/2306.01386/) (2023)
- [Enhancing Dialogue State Tracking Models through LLM-backed User-Agents Simulation](https://mlsanity.com/paper/2405.13037/) (2024)
- ["Do you follow me?": A Survey of Recent Approaches in Dialogue State Tracking](https://mlsanity.com/paper/2207.14627/) (2022)
- [Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2204.06677/) (2022)
- [Robust Dialogue State Tracking with Weak Supervision and Sparse Data](https://mlsanity.com/paper/2202.03354/) (2022)
- [Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues](https://mlsanity.com/paper/2111.02574/) (2021)
- [Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking](https://mlsanity.com/paper/2306.00434/) (2023)
- [Cyber Sentinel: Exploring Conversational Agents in Streamlining Security Tasks with GPT-4](https://mlsanity.com/paper/2309.16422/) (2023)
- [Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performance](https://mlsanity.com/paper/2109.04349/) (2021)
- [Parameter-Efficient Low-Resource Dialogue State Tracking by Prompt Tuning](https://mlsanity.com/paper/2301.10915/) (2023)
- [Reasoning-Enhanced Object-Centric Learning for Videos](https://mlsanity.com/paper/2403.15245/) (2024)
- [LUNA: Learning Slot-Turn Alignment for Dialogue State Tracking](https://mlsanity.com/paper/2205.02550/) (2022)
- [Mismatch between Multi-turn Dialogue and its Evaluation Metric in Dialogue State Tracking](https://mlsanity.com/paper/2203.03123/) (2022)
- [Diable: Efficient Dialogue State Tracking as Operations on Tables](https://mlsanity.com/paper/2305.17020/) (2023)
- [MoNET: Tackle State Momentum via Noise-Enhanced Training for Dialogue State Tracking](https://mlsanity.com/paper/2211.05503/) (2022)
- [What Did You Say? Task-Oriented Dialog Datasets Are Not Conversational!?](https://mlsanity.com/paper/2203.03431/) (2022)
- [Revisiting Conversation Discourse for Dialogue Disentanglement](https://mlsanity.com/paper/2306.03975/) (2023)
- [CSS: Combining Self-training and Self-supervised Learning for Few-shot Dialogue State Tracking](https://mlsanity.com/paper/2210.05146/) (2022)
- [Dialogue State Distillation Network with Inter-slot Contrastive Learning for Dialogue State Tracking](https://mlsanity.com/paper/2302.08220/) (2023)
- [DSTEA: Improving Dialogue State Tracking via Entity Adaptive Pre-training](https://mlsanity.com/paper/2207.03858/) (2022)
- [Turn-Level Active Learning for Dialogue State Tracking](https://mlsanity.com/paper/2310.14513/) (2023)
- [Multimodal Contextualized Semantic Parsing from Speech](https://mlsanity.com/paper/2406.06438/) (2024)
- [Know Your Mistakes: Towards Preventing Overreliance on Task-Oriented Conversational AI Through Accountability Modeling](https://mlsanity.com/paper/2501.10316/) (2025)
- [Dynamic Knowledge Fusion for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2603.10367/) (2026)

## Abstract (arXiv)

An indispensable component in task-oriented dialogue systems is the dialogue state tracker, which keeps track of users' intentions in the course of conversation. The typical approach towards this goal is to fill in multiple pre-defined slots that are essential to complete the task. Although various dialogue state tracking methods have been proposed in recent years, most of them predict the value of each slot separately and fail to consider the correlations among slots. In this paper, we propose a slot self-attention mechanism that can learn the slot correlations automatically. Specifically, a slot-token attention is first utilized to obtain slot-specific features from the dialogue context. Then a stacked slot self-attention is applied on these features to learn the correlations among slots. We conduct comprehensive experiments on two multi-domain task-oriented dialogue datasets, including MultiWOZ 2.0 and MultiWOZ 2.1. The experimental results demonstrate that our approach achieves state-of-the-art performance on both datasets, verifying the necessity and effectiveness of taking slot correlations into consideration.