# Few-Shot NLG with Pre-Trained Language Model

arXiv: https://arxiv.org/abs/1904.09521 · HTML: https://mlsanity.com/paper/1904.09521/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2019.

## Built on by (most cited first, 30 of 52)

- [A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems](https://mlsanity.com/paper/2402.18013/) (2024)
- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2105.10311/) (2021)
- [Plan-then-Generate: Controlled Data-to-Text Generation via Planning](https://mlsanity.com/paper/2108.13740/) (2021)
- [Controlling Hallucinations at Word Level in Data-to-Text Generation](https://mlsanity.com/paper/2102.02810/) (2021)
- [Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2105.03953/) (2021)
- [Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models](https://mlsanity.com/paper/2106.01623/) (2021)
- [On Training Instance Selection for Few-Shot Neural Text Generation](https://mlsanity.com/paper/2107.03176/) (2021)
- [Neural Data-to-Text Generation with LM-based Text Augmentation](https://mlsanity.com/paper/2102.03556/) (2021)
- [Learning to Transfer Prompts for Text Generation](https://mlsanity.com/paper/2205.01543/) (2022)
- [Few-Shot Table-to-Text Generation with Prototype Memory](https://mlsanity.com/paper/2108.12516/) (2021)
- [NER-BERT: A Pre-trained Model for Low-Resource Entity Tagging](https://mlsanity.com/paper/2112.00405/) (2021)
- [Neural Pipeline for Zero-Shot Data-to-Text Generation](https://mlsanity.com/paper/2203.16279/) (2022)
- [Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setup](https://mlsanity.com/paper/2112.06204/) (2021)
- [CrossCodeBench: Benchmarking Cross-Task Generalization of Source Code Models](https://mlsanity.com/paper/2302.04030/) (2023)
- [Improving the Sample Efficiency of Prompt Tuning with Domain Adaptation](https://mlsanity.com/paper/2210.02952/) (2022)
- [ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Models](https://mlsanity.com/paper/2210.04325/) (2022)
- [Few-shot learning for medical text: A systematic review](https://mlsanity.com/paper/2204.14081/) (2022)
- [Jointly Improving Language Understanding and Generation with Quality-Weighted Weak Supervision of Automatic Labeling](https://mlsanity.com/paper/2102.03551/) (2021)
- [Generative Conversational Networks](https://mlsanity.com/paper/2106.08484/) (2021)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [Leveraging Large (Visual) Language Models for Robot 3D Scene Understanding](https://mlsanity.com/paper/2209.05629/) (2022)
- [Search and Learn: Improving Semantic Coverage for Data-to-Text Generation](https://mlsanity.com/paper/2112.02770/) (2021)
- [What Makes Data-to-Text Generation Hard for Pretrained Language Models?](https://mlsanity.com/paper/2205.11505/) (2022)
- [AutoQGS: Auto-Prompt for Low-Resource Knowledge-based Question Generation from SPARQL](https://mlsanity.com/paper/2208.12461/) (2022)
- [A Comparative Analysis of Conversational Large Language Models in Knowledge-Based Text Generation](https://mlsanity.com/paper/2402.01495/) (2024)
- [Active Few-Shot Learning for Text Classification](https://mlsanity.com/paper/2502.18782/) (2025)
- [Towards objectively evaluating the quality of generated medical summaries](https://mlsanity.com/paper/2104.04412/) (2021)
- [The SelectGen Challenge: Finding the Best Training Samples for Few-Shot Neural Text Generation](https://mlsanity.com/paper/2108.06614/) (2021)

## Abstract (arXiv)

Neural-based end-to-end approaches to natural language generation (NLG) from structured data or knowledge are data-hungry, making their adoption for real-world applications difficult with limited data. In this work, we propose the new task of textitfew-shot natural language generation. Motivated by how humans tend to summarize tabular data, we propose a simple yet effective approach and show that it not only demonstrates strong performance but also provides good generalization across domains. The design of the model architecture is based on two aspects: content selection from input data and language modeling to compose coherent sentences, which can be acquired from prior knowledge. With just 200 training examples, across multiple domains, we show that our approach achieves very reasonable performances and outperforms the strongest baseline by an average of over 8.0 BLEU points improvement. Our code and data can be found at urlhttps://github.com/czyssrs/Few-Shot-NLG