MoNET: Tackle State Momentum via Noise-Enhanced Training for Dialogue State Tracking
Not on the map. Why?
Abstract (arXiv)
Dialogue state tracking (DST) aims to convert the dialogue history into dialogue states which consist of slot-value pairs. As condensed structural information memorizing all history information, the dialogue state in the last turn is typically adopted as the input for predicting the current state by DST models. However, these models tend to keep the predicted slot values unchanged, which is defined as state momentum in this paper. Specifically, the models struggle to update slot values that need to be changed and correct wrongly predicted slot values in the last turn. To this end, we propose MoNET to tackle state momentum via noise-enhanced training. First, the previous state of each turn in the training data is noised via replacing some of its slot values. Then, the noised previous state is used as the input to learn to predict the current state, improving the model's ability to update and correct slot values. Furthermore, a contrastive context matching framework is designed to narrow the representation distance between a state and its corresponding noised variant, which reduces the impact of noised state and makes the model better understand the dialogue history. Experimental results on MultiWOZ datasets show that MoNET outperforms previous DST methods. Ablations and analysis verify the effectiveness of MoNET in alleviating state momentum and improving anti-noise ability.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 43 of its 63 references that are in the map. First posted on arXiv 10 Nov 2022.
Builds on
43 of 63 references.
References · 43
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
- 2018Deep contextualized word representationscited by 2,285
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
- 2020A Simple Language Model for Task-Oriented Dialoguecited by 331
- 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
- 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
31 more
- 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 215
- 2016Neural Belief Tracker: Data-Driven Dialogue State Trackingcited by 142
- 2021Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue Systemcited by 139
- 2020TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Trackingcited by 130
- 2020UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2cited by 111
- 2020MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systemscited by 103
- 2021Dialogue State Tracking with a Language Model using Schema-Driven Promptingcited by 94
- 2021MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluationcited by 86
- 2019Efficient Dialogue State Tracking by Selectively Overwriting Memorycited by 76
- 2019Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Trackingcited by 75
- 2019SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Trackingcited by 63
- 2021CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understandingcited by 50
- 2020CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackerscited by 43
- 2020A Sequence-to-Sequence Approach to Dialogue State Trackingcited by 43
- 2021Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialog State Trackingcited by 35
- 2018Towards Universal Dialogue State Trackingcited by 33
- 2018An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Trackingcited by 27
- 2020Efficient Context and Schema Fusion Networks for Multi-Domain Dialogue State Trackingcited by 27
- 2021Slot Self-Attentive Dialogue State Trackingcited by 27
- 2020Contrastive Self-Supervised Learning for Commonsense Reasoningcited by 25
- 2017Scalable Multi-Domain Dialogue State Trackingcited by 22
- 2021Amendable Generation for Dialogue State Trackingcited by 21
- 2021Knowledge-Aware Graph-Enhanced GPT-2 for Dialogue State Trackingcited by 20
- 2022BORT: Back and Denoising Reconstruction for End-to-End Task-Oriented Dialogcited by 20
- 2022Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Trackingcited by 18
- 2020Group-wise Contrastive Learning for Neural Dialogue Generationcited by 15
- 2022ASSIST: Towards Label Noise-Robust Dialogue State Trackingcited by 11
- 2020Parallel Interactive Networks for Multi-Domain Dialogue State Generationcited by 6
- 2022LUNA: Learning Slot-Turn Alignment for Dialogue State Trackingcited by 6
- 2014Pragmatic Neural Language Modelling in Machine Translationcited by 2
- 2022CSS: Combining Self-training and Self-supervised Learning for Few-shot Dialogue State Trackingcited by 1
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.