mlsanityData to 7 Oct 2026

Recent Advances in Neural Text Generation: A Task-Agnostic Survey

arXiv 2203.03047 ↗First posted 6 Mar 2022cs.CL cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

In recent years, considerable research has been dedicated to the application of neural models in the field of natural language generation (NLG). The primary objective is to generate text that is both linguistically natural and human-like, while also exerting control over the generation process. This paper offers a comprehensive and task-agnostic survey of the recent advancements in neural text generation. These advancements have been facilitated through a multitude of developments, which we categorize into four key areas: data construction, neural frameworks, training and inference strategies, and evaluation metrics. By examining these different aspects, we aim to provide a holistic overview of the progress made in the field. Furthermore, we explore the future directions for the advancement of neural text generation, which encompass the utilization of neural pipelines and the incorporation of background knowledge. These avenues present promising opportunities to further enhance the capabilities of NLG systems. Overall, this survey serves to consolidate the current state of the art in neural text generation and highlights potential avenues for future research and development in this dynamic field.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 174 of its 270 references that are in the map. First posted on arXiv 6 Mar 2022.

Builds on

174 of 270 references.

References · 174

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2013Auto-Encoding Variational Bayescited by 3,980
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  9. 2018Deep contextualized word representationscited by 2,285
  10. 2020Knowledge Distillation: A Surveycited by 1,578
  11. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  12. 2018Hierarchical Neural Story Generationcited by 1,182
38 more
  1. 2018Self-Attention with Relative Position Representationscited by 1,112
  2. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  3. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  4. 2018Self-Attention Generative Adversarial Networkscited by 809
  5. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  6. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  7. 2018Object Hallucination in Image Captioningcited by 661
  8. 2015Generating Sentences from a Continuous Spacecited by 607
  9. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  10. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  11. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  12. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
  13. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  14. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  15. 2017Wasserstein Auto-Encoderscited by 371
  16. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  17. 2014Memory Networkscited by 354
  18. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  19. 2020A Survey on Neural Network Interpretabilitycited by 291
  20. 2016Deep Reinforcement Learning for Dialogue Generationcited by 286
  21. 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
  22. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
  23. 2020Evaluation of Text Generation: A Surveycited by 275
  24. 2017Toward Controlled Generation of Textcited by 271
  25. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 267
  26. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 252
  27. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  28. 2017Why We Need New Evaluation Metrics for NLGcited by 232
  29. 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 232
  30. 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 221
  31. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 221
  32. 2019Recurrent Independent Mechanismscited by 219
  33. 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
  34. 2019SemEval-2017 Task 4: Sentiment Analysis in Twittercited by 209
  35. 2015Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Modelscited by 207
  36. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  37. 2020The Next Decade in AI: Four Steps Towards Robust Artificial Intelligencecited by 194
  38. 2021Does Knowledge Distillation Really Work?cited by 187

50 most cited of 174 All 174 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 8; 2024: 2.

14 papers.

Sort:

By area:

  1. 2023FrameBERT: Conceptual Metaphor Detection with Frame Embedding Learningcited by 18
  2. 2022EtriCA: Event-Triggered Context-Aware Story Generation Augmented by Cross Attentioncited by 17
  3. 2023"Why is this misleading?": Detecting News Headline Hallucinations with Explanationscited by 17
  4. 2023Enhancing Dialogue Generation via Dynamic Graph Knowledge Aggregationcited by 16
  5. 2022NGEP: A Graph-based Event Planning Framework for Story Generationcited by 13
  6. 2022Improving Chinese Story Generation via Awareness of Syntactic Dependencies and Semanticscited by 10
  7. 2023Improving Biomedical Abstractive Summarisation with Knowledge Aggregation from Citation Paperscited by 9
  8. 2022Terminology-aware Medical Dialogue Generationcited by 8
  9. 2023CTRLStruct: Dialogue Structure Learning for Open-Domain Response Generationcited by 8
  10. 2023CADGE: Context-Aware Dialogue Generation Enhanced with Graph-Structured Knowledge Aggregationcited by 8
  11. 2023A Cross-Attention Augmented Model for Event-Triggered Context-Aware Story Generationcited by 5
  12. 2024Causal prompting model-based offline reinforcement learningcited by 1
2 more
  1. 2024Beyond Generative Artificial Intelligence: Roadmap for Natural Language Generationcited by 1
  2. 2023ACL Anthology Helper: A Tool to Retrieve and Manage Literature from ACL Anthologycited by 0