# Natural Language Generation by Hierarchical Decoding with Linguistic Patterns

arXiv: https://arxiv.org/abs/1808.02747 · HTML: https://mlsanity.com/paper/1808.02747/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2018.

## Built on by (most cited first, 11 of 11)

- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [AGGGEN: Ordering and Aggregating while Generating](https://mlsanity.com/paper/2106.05580/) (2021)
- [Multi-View Feature Representation for Dialogue Generation with Bidirectional Distillation](https://mlsanity.com/paper/2102.10780/) (2021)
- [TREND: Trigger-Enhanced Relation-Extraction Network for Dialogues](https://mlsanity.com/paper/2108.13811/) (2021)
- [Diversifying Neural Text Generation with Part-of-Speech Guided Softmax and Sampling](https://mlsanity.com/paper/2105.03641/) (2021)
- [Adaptive Natural Language Generation for Task-oriented Dialogue via Reinforcement Learning](https://mlsanity.com/paper/2209.07873/) (2022)
- [Improving Computer Generated Dialog with Auxiliary Loss Functions and Custom Evaluation Metrics](https://mlsanity.com/paper/2106.02516/) (2021)
- [An empirical assessment of deep learning approaches to task-oriented dialog management](https://mlsanity.com/paper/2108.03478/) (2021)
- [Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems](https://mlsanity.com/paper/2212.09252/) (2022)
- [Requirements Elicitation in Cognitive Service for Recommendation](https://mlsanity.com/paper/2203.14958/) (2022)
- [Making Language Model a Hierarchical Classifier](https://mlsanity.com/paper/2507.12930/) (2025)

## Abstract (arXiv)

Natural language generation (NLG) is a critical component in spoken dialogue systems. Classic NLG can be divided into two phases: (1) sentence planning: deciding on the overall sentence structure, (2) surface realization: determining specific word forms and flattening the sentence structure into a string. Many simple NLG models are based on recurrent neural networks (RNN) and sequence-to-sequence (seq2seq) model, which basically contains an encoder-decoder structure; these NLG models generate sentences from scratch by jointly optimizing sentence planning and surface realization using a simple cross entropy loss training criterion. However, the simple encoder-decoder architecture usually suffers from generating complex and long sentences, because the decoder has to learn all grammar and diction knowledge. This paper introduces a hierarchical decoding NLG model based on linguistic patterns in different levels, and shows that the proposed method outperforms the traditional one with a smaller model size. Furthermore, the design of the hierarchical decoding is flexible and easily-extensible in various NLG systems.