# Neural Language Generation: Formulation, Methods, and Evaluation

arXiv: https://arxiv.org/abs/2007.15780 · HTML: https://mlsanity.com/paper/2007.15780/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2020.

## Built on by (most cited first, 13 of 13)

- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation](https://mlsanity.com/paper/2109.13296/) (2021)
- [Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks](https://mlsanity.com/paper/2310.19740/) (2023)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [MVP: Multi-task Supervised Pre-training for Natural Language Generation](https://mlsanity.com/paper/2206.12131/) (2022)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [Automated Evaluation of Personalized Text Generation using Large Language Models](https://mlsanity.com/paper/2310.11593/) (2023)
- [Creative Data Generation: A Review Focusing on Text and Poetry](https://mlsanity.com/paper/2305.08493/) (2023)
- [PCAE: A Framework of Plug-in Conditional Auto-Encoder for Controllable Text Generation](https://mlsanity.com/paper/2210.03496/) (2022)
- [COMPASS: a Creative Support System that Alerts Novelists to the Unnoticed Missing Contents](https://mlsanity.com/paper/2202.13151/) (2022)
- [Augmenting Textual Generation via Topology Aware Retrieval](https://mlsanity.com/paper/2405.17602/) (2024)
- [An Overview on Controllable Text Generation via Variational Auto-Encoders](https://mlsanity.com/paper/2211.07954/) (2022)
- [Bangla Key2Text: Text Generation from Keywords for a Low Resource Language](https://mlsanity.com/paper/2604.19508/) (2026)

## Abstract (arXiv)

Recent advances in neural network-based generative modeling have reignited the hopes in having computer systems capable of seamlessly conversing with humans and able to understand natural language. Neural architectures have been employed to generate text excerpts to various degrees of success, in a multitude of contexts and tasks that fulfil various user needs. Notably, high capacity deep learning models trained on large scale datasets demonstrate unparalleled abilities to learn patterns in the data even in the lack of explicit supervision signals, opening up a plethora of new possibilities regarding producing realistic and coherent texts. While the field of natural language generation is evolving rapidly, there are still many open challenges to address. In this survey we formally define and categorize the problem of natural language generation. We review particular application tasks that are instantiations of these general formulations, in which generating natural language is of practical importance. Next we include a comprehensive outline of methods and neural architectures employed for generating diverse texts. Nevertheless, there is no standard way to assess the quality of text produced by these generative models, which constitutes a serious bottleneck towards the progress of the field. To this end, we also review current approaches to evaluating natural language generation systems. We hope this survey will provide an informative overview of formulations, methods, and assessments of neural natural language generation.