# State Value Generation with Prompt Learning and Self-Training for Low-Resource Dialogue State Tracking

arXiv: https://arxiv.org/abs/2401.16862 · HTML: https://mlsanity.com/paper/2401.16862/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 23 references that are in the map.
- First posted on arXiv 30 Jan 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [Revisiting Self-Training for Neural Sequence Generation](https://mlsanity.com/paper/1909.13788/) (2019)
- [Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System](https://mlsanity.com/paper/2109.14739/) (2021)
- [Dialogue State Tracking with a Language Model using Schema-Driven Prompting](https://mlsanity.com/paper/2109.07506/) (2021)
- [Zero-Shot Dialogue State Tracking via Cross-Task Transfer](https://mlsanity.com/paper/2109.04655/) (2021)
- [In-Context Learning for Few-Shot Dialogue State Tracking](https://mlsanity.com/paper/2203.08568/) (2022)
- [From Machine Reading Comprehension to Dialogue State Tracking: Bridging the Gap](https://mlsanity.com/paper/2004.05827/) (2020)
- [Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking](https://mlsanity.com/paper/2203.01552/) (2022)
- [Improving Compositional Generalization with Self-Training for Data-to-Text Generation](https://mlsanity.com/paper/2110.08467/) (2021)
- [Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systems](https://mlsanity.com/paper/2108.12589/) (2021)
- [Improving Limited Labeled Dialogue State Tracking with Self-Supervision](https://mlsanity.com/paper/2010.13920/) (2020)
- [Few Shot Dialogue State Tracking using Meta-learning](https://mlsanity.com/paper/2101.06779/) (2021)
- [Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking](https://mlsanity.com/paper/2302.05932/) (2023)
- [Dual Slot Selector via Local Reliability Verification for Dialogue State Tracking](https://mlsanity.com/paper/2107.12578/) (2021)

## Abstract (arXiv)

Recently, low-resource dialogue state tracking (DST) has received increasing attention. First obtaining state values then based on values to generate slot types has made great progress in this task. However, obtaining state values is still an under-studied problem. Existing extraction-based approaches cannot capture values that require the understanding of context and are not generalizable either. To address these issues, we propose a novel State VAlue Generation based framework (SVAG), decomposing DST into state value generation and domain slot generation. Specifically, we propose to generate state values and use self-training to further improve state value generation. Moreover, we design an estimator aiming at detecting incomplete generation and incorrect generation for pseudo-labeled data selection during self-training. Experimental results on the MultiWOZ 2.1 dataset show that our method which has only less than 1 billion parameters achieves state-of-the-art performance under the data ratio settings of 5%, 10%, and 25% when limited to models under 100 billion parameters. Compared to models with more than 100 billion parameters, SVAG still reaches competitive results.