# A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions

arXiv: https://arxiv.org/abs/1906.00500 · HTML: https://mlsanity.com/paper/1906.00500/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2019.

## Built on by (most cited first, 26 of 26)

- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods](https://mlsanity.com/paper/2210.07321/) (2022)
- [TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations](https://mlsanity.com/paper/2207.04154/) (2022)
- [End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2311.09008/) (2023)
- [Recent Advances in Neural Text Generation: A Task-Agnostic Survey](https://mlsanity.com/paper/2203.03047/) (2022)
- [A Survey of Natural Language Generation](https://mlsanity.com/paper/2112.11739/) (2021)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [A Survey of Personality, Persona, and Profile in Conversational Agents and Chatbots](https://mlsanity.com/paper/2401.00609/) (2023)
- [Beyond Traditional Teaching: The Potential of Large Language Models and Chatbots in Graduate Engineering Education](https://mlsanity.com/paper/2309.13059/) (2023)
- [Exploring Conversational Agents as an Effective Tool for Measuring Cognitive Biases in Decision-Making](https://mlsanity.com/paper/2401.06686/) (2024)
- [FCM: A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning](https://mlsanity.com/paper/2109.10510/) (2021)
- [Task-oriented Dialogue Systems: performance vs. quality-optima, a review](https://mlsanity.com/paper/2112.11176/) (2021)
- [A Survey on Artificial Intelligence for Source Code: A Dialogue Systems Perspective](https://mlsanity.com/paper/2202.04847/) (2022)
- [Towards a Neural Era in Dialogue Management for Collaboration: A Literature Survey](https://mlsanity.com/paper/2307.09021/) (2023)
- [Enhancing Dialogue Generation in Werewolf Game Through Situation Analysis and Persuasion Strategies](https://mlsanity.com/paper/2408.16586/) (2024)
- [Introducing Brain-like Concepts to Embodied Hand-crafted Dialog Management System](https://mlsanity.com/paper/2406.08996/) (2024)
- [Beyond Generative Artificial Intelligence: Roadmap for Natural Language Generation](https://mlsanity.com/paper/2407.10554/) (2024)
- [From Intents to Conversations: Generating Intent-Driven Dialogues with Contrastive Learning for Multi-Turn Classification](https://mlsanity.com/paper/2411.14252/) (2024)
- [Talking with Machines: A Comprehensive Survey of Emergent Dialogue Systems](https://mlsanity.com/paper/2305.16324/) (2023)
- [Experimental Evaluation of Machine Learning Models for Goal-oriented Customer Service Chatbot with Pipeline Architecture](https://mlsanity.com/paper/2409.18568/) (2024)
- [Unstructured Text Enhanced Open-domain Dialogue System: A Systematic Survey](https://mlsanity.com/paper/2411.09166/) (2024)
- [Language Generation for Broad-Coverage, Explainable Cognitive Systems](https://mlsanity.com/paper/2201.10422/) (2022)
- [Requirements Elicitation in Cognitive Service for Recommendation](https://mlsanity.com/paper/2203.14958/) (2022)
- [Collocation2Text: Controllable Text Generation from Guide Phrases in Russian](https://mlsanity.com/paper/2206.09248/) (2022)
- [A Multi-view Discourse Framework for Integrating Semantic and Syntactic Features in Dialog Agents](https://mlsanity.com/paper/2504.09073/) (2025)
- [From Instruction to Output: The Role of Prompting in Modern NLG](https://mlsanity.com/paper/2602.11179/) (2026)

## Abstract (arXiv)

One of the hardest problems in the area of Natural Language Processing and Artificial Intelligence is automatically generating language that is coherent and understandable to humans. Teaching machines how to converse as humans do falls under the broad umbrella of Natural Language Generation. Recent years have seen unprecedented growth in the number of research articles published on this subject in conferences and journals both by academic and industry researchers. There have also been several workshops organized alongside top-tier NLP conferences dedicated specifically to this problem. All this activity makes it hard to clearly define the state of the field and reason about its future directions. In this work, we provide an overview of this important and thriving area, covering traditional approaches, statistical approaches and also approaches that use deep neural networks. We provide a comprehensive review towards building open domain dialogue systems, an important application of natural language generation. We find that, predominantly, the approaches for building dialogue systems use seq2seq or language models architecture. Notably, we identify three important areas of further research towards building more effective dialogue systems: 1) incorporating larger context, including conversation context and world knowledge; 2) adding personae or personality in the NLG system; and 3) overcoming dull and generic responses that affect the quality of system-produced responses. We provide pointers on how to tackle these open problems through the use of cognitive architectures that mimic human language understanding and generation capabilities.