# Consistency and Coherency Enhanced Story Generation

arXiv: https://arxiv.org/abs/2010.08822 · HTML: https://mlsanity.com/paper/2010.08822/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2020.

## Built on by (most cited first, 5 of 5)

- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [DAGN: Discourse-Aware Graph Network for Logical Reasoning](https://mlsanity.com/paper/2103.14349/) (2021)
- [Towards Coherent and Consistent Use of Entities in Narrative Generation](https://mlsanity.com/paper/2202.01709/) (2022)
- [Little Red Riding Hood Goes Around the Globe:Crosslingual Story Planning and Generation with Large Language Models](https://mlsanity.com/paper/2212.10471/) (2022)
- [Automatic Generation of German Drama Texts Using Fine Tuned GPT-2 Models](https://mlsanity.com/paper/2301.03119/) (2023)

## Abstract (arXiv)

Story generation is a challenging task, which demands to maintain consistency of the plots and characters throughout the story. Previous works have shown that GPT2, a large-scale language model, has achieved good performance on story generation. However, we observe that several serious issues still exist in the stories generated by GPT2 which can be categorized into two folds: consistency and coherency. In terms of consistency, on one hand, GPT2 cannot guarantee the consistency of the plots explicitly. On the other hand, the generated stories usually contain coreference errors. In terms of coherency, GPT2 does not take account of the discourse relations between sentences of stories directly. To enhance the consistency and coherency of the generated stories, we propose a two-stage generation framework, where the first stage is to organize the story outline which depicts the story plots and events, and the second stage is to expand the outline into a complete story. Therefore the plots consistency can be controlled and guaranteed explicitly. In addition, coreference supervision signals are incorporated to reduce coreference errors and improve the coreference consistency. Moreover, we design an auxiliary task of discourse relation modeling to improve the coherency of the generated stories. Experimental results on a story dataset show that our model outperforms the baseline approaches in terms of both automatic metrics and human evaluation.