# A Good Sample is Hard to Find: Noise Injection Sampling and Self-Training for Neural Language Generation Models

arXiv: https://arxiv.org/abs/1911.03373 · HTML: https://mlsanity.com/paper/1911.03373/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2019.

## Built on by (most cited first, 17 of 17)

- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [Text Generation by Learning from Demonstrations](https://mlsanity.com/paper/2009.07839/) (2020)
- [Improving Factual Consistency of Abstractive Summarization via Question Answering](https://mlsanity.com/paper/2105.04623/) (2021)
- [Rethinking Semi-supervised Learning with Language Models](https://mlsanity.com/paper/2305.13002/) (2023)
- [Recent Advances in Neural Text Generation: A Task-Agnostic Survey](https://mlsanity.com/paper/2203.03047/) (2022)
- [Attention Is Indeed All You Need: Semantically Attention-Guided Decoding for Data-to-Text NLG](https://mlsanity.com/paper/2109.07043/) (2021)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [Retrieve, Caption, Generate: Visual Grounding for Enhancing Commonsense in Text Generation Models](https://mlsanity.com/paper/2109.03892/) (2021)
- [AGGGEN: Ordering and Aggregating while Generating](https://mlsanity.com/paper/2106.05580/) (2021)
- [Jurassic is (almost) All You Need: Few-Shot Meaning-to-Text Generation for Open-Domain Dialogue](https://mlsanity.com/paper/2110.08094/) (2021)
- [Controllable Generation of Dialogue Acts for Dialogue Systems via Few-Shot Response Generation and Ranking](https://mlsanity.com/paper/2307.14440/) (2023)
- [A Systematic Review of Data-to-Text NLG](https://mlsanity.com/paper/2402.08496/) (2024)
- [ArcSin: Adaptive ranged cosine Similarity injected noise for Language-Driven Visual Tasks](https://mlsanity.com/paper/2402.17298/) (2024)
- [Self-training with Two-phase Self-augmentation for Few-shot Dialogue Generation](https://mlsanity.com/paper/2205.09661/) (2022)
- [Neural Data-to-Text Generation Based on Small Datasets: Comparing the Added Value of Two Semi-Supervised Learning Approaches on Top of a Large Language Model](https://mlsanity.com/paper/2207.06839/) (2022)
- [Self-training from Self-memory in Data-to-text Generation](https://mlsanity.com/paper/2401.10567/) (2024)
- [Harnessing the Power of Noise: A Survey of Techniques and Applications](https://mlsanity.com/paper/2410.06348/) (2024)

## Abstract (arXiv)

Deep neural networks (DNN) are quickly becoming the de facto standard modeling method for many natural language generation (NLG) tasks. In order for such models to truly be useful, they must be capable of correctly generating utterances for novel meaning representations (MRs) at test time. In practice, even sophisticated DNNs with various forms of semantic control frequently fail to generate utterances faithful to the input MR. In this paper, we propose an architecture agnostic self-training method to sample novel MR/text utterance pairs to augment the original training data. Remarkably, after training on the augmented data, even simple encoder-decoder models with greedy decoding are capable of generating semantically correct utterances that are as good as state-of-the-art outputs in both automatic and human evaluations of quality.