# CSS: Combining Self-training and Self-supervised Learning for Few-shot Dialogue State Tracking

arXiv: https://arxiv.org/abs/2210.05146 · HTML: https://mlsanity.com/paper/2210.05146/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 39 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://mlsanity.com/paper/2105.11741/) (2021)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [Neural Belief Tracker: Data-Driven Dialogue State Tracking](https://mlsanity.com/paper/1606.03777/) (2016)
- [Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System](https://mlsanity.com/paper/2109.14739/) (2021)
- [UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2](https://mlsanity.com/paper/2012.03539/) (2020)
- [MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2009.12005/) (2020)
- [Efficient Dialogue State Tracking by Selectively Overwriting Memory](https://mlsanity.com/paper/1911.03906/) (2019)
- [Zero-Shot Dialogue State Tracking via Cross-Task Transfer](https://mlsanity.com/paper/2109.04655/) (2021)
- [From Machine Reading Comprehension to Dialogue State Tracking: Bridging the Gap](https://mlsanity.com/paper/2004.05827/) (2020)
- [An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking](https://mlsanity.com/paper/1805.01555/) (2018)
- [Slot Self-Attentive Dialogue State Tracking](https://mlsanity.com/paper/2101.09374/) (2021)
- [Contrastive Self-Supervised Learning for Commonsense Reasoning](https://mlsanity.com/paper/2005.00669/) (2020)
- [Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking](https://mlsanity.com/paper/2203.01552/) (2022)
- [BORT: Back and Denoising Reconstruction for End-to-End Task-Oriented Dialog](https://mlsanity.com/paper/2205.02471/) (2022)
- [Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systems](https://mlsanity.com/paper/2108.12589/) (2021)
- [Explicit State Tracking with Semi-Supervision for Neural Dialogue Generation](https://mlsanity.com/paper/1808.10596/) (2018)
- [Self-Supervised Dialogue Learning](https://mlsanity.com/paper/1907.00448/) (2019)
- [Improving Limited Labeled Dialogue State Tracking with Self-Supervision](https://mlsanity.com/paper/2010.13920/) (2020)
- [Group-wise Contrastive Learning for Neural Dialogue Generation](https://mlsanity.com/paper/2009.07543/) (2020)
- [ASSIST: Towards Label Noise-Robust Dialogue State Tracking](https://mlsanity.com/paper/2202.13024/) (2022)
- [Combining Self-Training and Self-Supervised Learning for Unsupervised Disfluency Detection](https://mlsanity.com/paper/2010.15360/) (2020)

## Built on by (most cited first, 1 of 1)

- [MoNET: Tackle State Momentum via Noise-Enhanced Training for Dialogue State Tracking](https://mlsanity.com/paper/2211.05503/) (2022)

## Abstract (arXiv)

Few-shot dialogue state tracking (DST) is a realistic problem that trains the DST model with limited labeled data. Existing few-shot methods mainly transfer knowledge learned from external labeled dialogue data (e.g., from question answering, dialogue summarization, machine reading comprehension tasks, etc.) into DST, whereas collecting a large amount of external labeled data is laborious, and the external data may not effectively contribute to the DST-specific task. In this paper, we propose a few-shot DST framework called CSS, which Combines Self-training and Self-supervised learning methods. The unlabeled data of the DST task is incorporated into the self-training iterations, where the pseudo labels are predicted by a DST model trained on limited labeled data in advance. Besides, a contrastive self-supervised method is used to learn better representations, where the data is augmented by the dropout operation to train the model. Experimental results on the MultiWOZ dataset show that our proposed CSS achieves competitive performance in several few-shot scenarios.