mlsanityData to 7 Oct 2026

A Survey of Natural Language Generation

arXiv 2112.11739 ↗First posted 22 Dec 2021cs.CL cs.AI cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

This paper offers a comprehensive review of the research on Natural Language Generation (NLG) over the past two decades, especially in relation to data-to-text generation and text-to-text generation deep learning methods, as well as new applications of NLG technology. This survey aims to (a) give the latest synthesis of deep learning research on the NLG core tasks, as well as the architectures adopted in the field; (b) detail meticulously and comprehensively various NLG tasks and datasets, and draw attention to the challenges in NLG evaluation, focusing on different evaluation methods and their relationships; (c) highlight some future emphasis and relatively recent research issues that arise due to the increasing synergy between NLG and other artificial intelligence areas, such as computer vision, text and computational creativity.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 107 of its 169 references that are in the map. First posted on arXiv 22 Dec 2021.

Builds on

107 of 169 references.

References · 107

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  7. 2017Graph Attention Networkscited by 8,854
  8. 2017Inductive Representation Learning on Large Graphscited by 6,284
  9. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  10. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  11. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  12. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
38 more
  1. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  2. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  3. 2015VQA: Visual Question Answeringcited by 2,489
  4. 2018Deep contextualized word representationscited by 2,285
  5. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  6. 2016Variational Graph Auto-Encoderscited by 1,373
  7. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  8. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  9. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  10. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  11. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  12. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  13. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  14. 2015Teaching Machines to Read and Comprehendcited by 1,039
  15. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  16. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  17. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  18. 2015Gated Graph Sequence Neural Networkscited by 792
  19. 2017Dense-Captioning Events in Videoscited by 789
  20. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  21. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  22. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  23. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
  24. 2016Self-critical Sequence Training for Image Captioningcited by 544
  25. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
  26. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  27. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  28. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  29. 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 483
  30. 2015Attention-Based Models for Speech Recognitioncited by 400
  31. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 394
  32. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 364
  33. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  34. 2020Evaluation of Text Generation: A Surveycited by 275
  35. 2020ActBERT: Learning Global-Local Video-Text Representationscited by 267
  36. 2016Image Captioning with Semantic Attentioncited by 260
  37. 2020UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generationcited by 258
  38. 2018Auto-Encoding Scene Graphs for Image Captioningcited by 254

50 most cited of 107 All 107 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 1; 2025: 4; 2026: 1.

13 papers.

Sort:

By area:

  1. 2022TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversationscited by 65
  2. 2023Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Designcited by 22
  3. 2023GPT-Based Models Meet Simulation: How to Efficiently Use Large-Scale Pre-Trained Language Models Across Simulation Taskscited by 14
  4. 2023Correct Like Humans: Progressive Learning Framework for Chinese Text Error Correctioncited by 13
  5. 2024Script-Strategy Aligned Generation: Aligning LLMs with Expert-Crafted Dialogue Scripts and Therapeutic Strategies for Psychotherapycited by 7
  6. 2023From Retrieval to Generation: Efficient and Effective Entity Set Expansioncited by 3
  7. 2025From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development -- An Opinion Papercited by 3
  8. 2025Natural Language Generation in Healthcare: A Review of Methods and Applicationscited by 3
  9. 2022A Curriculum Learning Approach for Multi-domain Text Classification Using Keyword weight Rankingcited by 2
  10. 2022E2E Refined Datasetcited by 0
  11. 2025LaQual: An Automated Framework for LLM App Quality Evaluationcited by 0
  12. 2025Agentic Meta-Orchestrator for Multi-task Copilotscited by 0
1 more
  1. 2026Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Modelscited by 0