# AUGNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation

arXiv: https://arxiv.org/abs/2106.05589 · HTML: https://mlsanity.com/paper/2106.05589/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 29 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1508.01745/) (2015)
- [Augmenting Data with Mixup for Sentence Classification: An Empirical Study](https://mlsanity.com/paper/1905.08941/) (2019)
- [Self-training Improves Pre-training for Natural Language Understanding](https://mlsanity.com/paper/2010.02194/) (2020)
- [Few-shot Natural Language Generation for Task-Oriented Dialog](https://mlsanity.com/paper/2002.12328/) (2020)
- [Continual Learning in Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2012.15504/) (2020)
- [A Repository of Conversational Datasets](https://mlsanity.com/paper/1904.06472/) (2019)
- [Learning Data Manipulation for Augmentation and Weighting](https://mlsanity.com/paper/1910.12795/) (2019)
- [Paraphrase Augmented Task-Oriented Dialog Generation](https://mlsanity.com/paper/2004.07462/) (2020)
- [Pre-trained Language Model Representations for Language Generation](https://mlsanity.com/paper/1903.09722/) (2019)
- [Schema-Guided Dialogue State Tracking Task at DSTC8](https://mlsanity.com/paper/2002.01359/) (2020)
- [Sequence-to-Sequence Generation for Spoken Dialogue via Deep Syntax Trees and Strings](https://mlsanity.com/paper/1606.05491/) (2016)
- [Multi-domain Neural Network Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1603.01232/) (2016)
- [Data Augmentation with Atomic Templates for Spoken Language Understanding](https://mlsanity.com/paper/1908.10770/) (2019)
- [Effective Data Augmentation Approaches to End-to-End Task-Oriented Dialogue](https://mlsanity.com/paper/1912.02478/) (2019)
- [Neural-based Natural Language Generation in Dialogue using RNN Encoder-Decoder with Semantic Aggregation](https://mlsanity.com/paper/1706.06714/) (2017)
- [Natural Language Generation for Spoken Dialogue System using RNN Encoder-Decoder Networks](https://mlsanity.com/paper/1706.00139/) (2017)

## Built on by (most cited first, 17 of 17)

- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [Data Augmentation Approaches in Natural Language Processing: A Survey](https://mlsanity.com/paper/2110.01852/) (2021)
- [PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks](https://mlsanity.com/paper/2202.12499/) (2022)
- [A Survey of Data Synthesis Approaches](https://mlsanity.com/paper/2407.03672/) (2024)
- [Aspect-Based Sentiment Analysis with Explicit Sentiment Augmentations](https://mlsanity.com/paper/2312.10961/) (2023)
- [Incubating Text Classifiers Following User Instruction with Nothing but LLM](https://mlsanity.com/paper/2404.10877/) (2024)
- [Dual-level Mixup for Graph Few-shot Learning with Fewer Tasks](https://mlsanity.com/paper/2502.14158/) (2025)
- [Controllable Generation of Dialogue Acts for Dialogue Systems via Few-Shot Response Generation and Ranking](https://mlsanity.com/paper/2307.14440/) (2023)
- [DREAM: Improving Video-Text Retrieval Through Relevance-Based Augmentation Using Large Foundation Models](https://mlsanity.com/paper/2404.05083/) (2024)
- [Few-Shot Table-to-Text Generation with Prompt Planning and Knowledge Memorization](https://mlsanity.com/paper/2302.04415/) (2023)
- [Unified Conversational Models with System-Initiated Transitions between Chit-Chat and Task-Oriented Dialogues](https://mlsanity.com/paper/2307.01664/) (2023)
- [Contextual Data Augmentation for Task-Oriented Dialog Systems](https://mlsanity.com/paper/2310.10380/) (2023)
- [Self-training with Two-phase Self-augmentation for Few-shot Dialogue Generation](https://mlsanity.com/paper/2205.09661/) (2022)
- [Selective Token Generation for Few-shot Natural Language Generation](https://mlsanity.com/paper/2209.08206/) (2022)
- [Compositional Generalization for Data-to-Text Generation](https://mlsanity.com/paper/2312.02748/) (2023)
- [Reinforcement Learning for Few-Shot Text Generation Adaptation](https://mlsanity.com/paper/2111.11030/) (2021)
- [A Comprehensive Survey of Challenges and Opportunities of Few-Shot Learning Across Multiple Domains](https://mlsanity.com/paper/2504.04017/) (2025)

## Abstract (arXiv)

Natural Language Generation (NLG) is a key component in a task-oriented dialogue system, which converts the structured meaning representation (MR) to the natural language. For large-scale conversational systems, where it is common to have over hundreds of intents and thousands of slots, neither template-based approaches nor model-based approaches are scalable. Recently, neural NLGs started leveraging transfer learning and showed promising results in few-shot settings. This paper proposes AUGNLG, a novel data augmentation approach that combines a self-trained neural retrieval model with a few-shot learned NLU model, to automatically create MR-to-Text data from open-domain texts. The proposed system mostly outperforms the state-of-the-art methods on the FewShotWOZ data in both BLEU and Slot Error Rate. We further confirm improved results on the FewShotSGD data and provide comprehensive analysis results on key components of our system. Our code and data are available at https://github.com/XinnuoXu/AugNLG.