# Self-training with Two-phase Self-augmentation for Few-shot Dialogue Generation

arXiv: https://arxiv.org/abs/2205.09661 · HTML: https://mlsanity.com/paper/2205.09661/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 19 May 2022.

## Builds on (most cited first, 26 of 26)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [Characterizing and Avoiding Negative Transfer](https://mlsanity.com/paper/1811.09751/) (2018)
- [Revisiting Self-Training for Neural Sequence Generation](https://mlsanity.com/paper/1909.13788/) (2019)
- [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1508.01745/) (2015)
- [UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2](https://mlsanity.com/paper/2012.03539/) (2020)
- [Few-shot Natural Language Generation for Task-Oriented Dialog](https://mlsanity.com/paper/2002.12328/) (2020)
- [Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation](https://mlsanity.com/paper/2010.12836/) (2020)
- [Few-Shot NLG with Pre-Trained Language Model](https://mlsanity.com/paper/1904.09521/) (2019)
- [GenAug: Data Augmentation for Finetuning Text Generators](https://mlsanity.com/paper/2010.01794/) (2020)
- [On Training Instance Selection for Few-Shot Neural Text Generation](https://mlsanity.com/paper/2107.03176/) (2021)
- [Active Learning for Interactive Neural Machine Translation of Data Streams](https://mlsanity.com/paper/1807.11243/) (2018)
- [Improving Compositional Generalization with Self-Training for Data-to-Text Generation](https://mlsanity.com/paper/2110.08467/) (2021)
- [Multi-domain Neural Network Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1603.01232/) (2016)
- [Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systems](https://mlsanity.com/paper/2108.12589/) (2021)
- [A Good Sample is Hard to Find: Noise Injection Sampling and Self-Training for Neural Language Generation Models](https://mlsanity.com/paper/1911.03373/) (2019)
- [AUGNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation](https://mlsanity.com/paper/2106.05589/) (2021)

## Built on by (most cited first, 1 of 1)

- [Controllable Generation of Dialogue Acts for Dialogue Systems via Few-Shot Response Generation and Ranking](https://mlsanity.com/paper/2307.14440/) (2023)

## Abstract (arXiv)

In task-oriented dialogue systems, response generation from meaning representations (MRs) often suffers from limited training examples, due to the high cost of annotating MR-to-Text pairs. Previous works on self-training leverage fine-tuned conversational models to automatically generate pseudo-labeled MR-to-Text pairs for further fine-tuning. However, some self-augmented data may be noisy or uninformative for the model to learn from. In this work, we propose a two-phase self-augmentation procedure to generate high-quality pseudo-labeled MR-to-Text pairs: the first phase selects the most informative MRs based on model's prediction uncertainty; with the selected MRs, the second phase generates accurate responses by aggregating multiple perturbed latent representations from each MR. Empirical experiments on two benchmark datasets, FewShotWOZ and FewShotSGD, show that our method generally outperforms existing self-training methods on both automatic and human evaluations.