# Knowledge-Aware Graph-Enhanced GPT-2 for Dialogue State Tracking

arXiv: https://arxiv.org/abs/2104.04466 · HTML: https://mlsanity.com/paper/2104.04466/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 9 Apr 2021.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking](https://mlsanity.com/paper/2005.02877/) (2020)
- [UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2](https://mlsanity.com/paper/2012.03539/) (2020)
- [Efficient Dialogue State Tracking by Selectively Overwriting Memory](https://mlsanity.com/paper/1911.03906/) (2019)
- [Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking](https://mlsanity.com/paper/1910.03544/) (2019)
- [Message-Aware Graph Attention Networks for Large-Scale Multi-Robot Path Planning](https://mlsanity.com/paper/2011.13219/) (2020)
- [Multi-domain Dialogue State Tracking as Dynamic Knowledge Graph Enhanced Question Answering](https://mlsanity.com/paper/1911.06192/) (2019)
- [Efficient Context and Schema Fusion Networks for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2004.03386/) (2020)
- [A Contextual Hierarchical Attention Network with Adaptive Objective for Dialogue State Tracking](https://mlsanity.com/paper/2006.01554/) (2020)
- [Attention Guided Dialogue State Tracking with Sparse Supervision](https://mlsanity.com/paper/2101.11958/) (2021)

## Built on by (most cited first, 20 of 20)

- ["Do you follow me?": A Survey of Recent Approaches in Dialogue State Tracking](https://mlsanity.com/paper/2207.14627/) (2022)
- [EmoWOZ: A Large-Scale Corpus and Labelling Scheme for Emotion Recognition in Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2109.04919/) (2021)
- [Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2204.06677/) (2022)
- [Task-Oriented Dialogue System as Natural Language Generation](https://mlsanity.com/paper/2108.13679/) (2021)
- [Aligning Large Language Models to a Domain-specific Graph Database for NL2GQL](https://mlsanity.com/paper/2402.16567/) (2024)
- [Common Ground Tracking in Multimodal Dialogue](https://mlsanity.com/paper/2403.17284/) (2024)
- [ASSIST: Towards Label Noise-Robust Dialogue State Tracking](https://mlsanity.com/paper/2202.13024/) (2022)
- [Response-act Guided Reinforced Dialogue Generation for Mental Health Counseling](https://mlsanity.com/paper/2301.12729/) (2023)
- [A Dual Prompt Learning Framework for Few-Shot Dialogue State Tracking](https://mlsanity.com/paper/2201.05780/) (2022)
- [CADGE: Context-Aware Dialogue Generation Enhanced with Graph-Structured Knowledge Aggregation](https://mlsanity.com/paper/2305.06294/) (2023)
- [Multimodal Dialogue State Tracking](https://mlsanity.com/paper/2206.07898/) (2022)
- [Mismatch between Multi-turn Dialogue and its Evaluation Metric in Dialogue State Tracking](https://mlsanity.com/paper/2203.03123/) (2022)
- [MoNET: Tackle State Momentum via Noise-Enhanced Training for Dialogue State Tracking](https://mlsanity.com/paper/2211.05503/) (2022)
- [Knowledge-Aware Audio-Grounded Generative Slot Filling for Limited Annotated Data](https://mlsanity.com/paper/2307.01764/) (2023)
- [XQA-DST: Multi-Domain and Multi-Lingual Dialogue State Tracking](https://mlsanity.com/paper/2204.05895/) (2022)
- [Dialogue State Distillation Network with Inter-slot Contrastive Learning for Dialogue State Tracking](https://mlsanity.com/paper/2302.08220/) (2023)
- [Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challenges](https://mlsanity.com/paper/2608.17605/) (2026)
- [Turn-Level Active Learning for Dialogue State Tracking](https://mlsanity.com/paper/2310.14513/) (2023)
- [Beyond Ontology in Dialogue State Tracking for Goal-Oriented Chatbot](https://mlsanity.com/paper/2410.22767/) (2024)
- [Dynamic Knowledge Fusion for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2603.10367/) (2026)

## Abstract (arXiv)

Dialogue State Tracking is central to multi-domain task-oriented dialogue systems, responsible for extracting information from user utterances. We present a novel hybrid architecture that augments GPT-2 with representations derived from Graph Attention Networks in such a way to allow causal, sequential prediction of slot values. The model architecture captures inter-slot relationships and dependencies across domains that otherwise can be lost in sequential prediction. We report improvements in state tracking performance in MultiWOZ 2.0 against a strong GPT-2 baseline and investigate a simplified sparse training scenario in which DST models are trained only on session-level annotations but evaluated at the turn level. We further report detailed analyses to demonstrate the effectiveness of graph models in DST by showing that the proposed graph modules capture inter-slot dependencies and improve the predictions of values that are common to multiple domains.