# LongStory: Coherent, Complete and Length Controlled Long story Generation

arXiv: https://arxiv.org/abs/2311.15208 · HTML: https://mlsanity.com/paper/2311.15208/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 39 references that are in the map.
- First posted on arXiv 26 Nov 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021)
- [Plan-And-Write: Towards Better Automatic Storytelling](https://mlsanity.com/paper/1811.05701/) (2018)
- [Re3: Generating Longer Stories With Recursive Reprompting and Revision](https://mlsanity.com/paper/2210.06774/) (2022)
- [BookSum: A Collection of Datasets for Long-form Narrative Summarization](https://mlsanity.com/paper/2105.08209/) (2021)
- [Strategies for Structuring Story Generation](https://mlsanity.com/paper/1902.01109/) (2019)
- [How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN](https://mlsanity.com/paper/2111.09509/) (2021)
- [A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation](https://mlsanity.com/paper/2001.05139/) (2020)
- [Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1907.05774/) (2019)
- [PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking](https://mlsanity.com/paper/2004.14967/) (2020)
- [Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generation](https://mlsanity.com/paper/2208.11646/) (2022)
- [Consistency of a Recurrent Language Model With Respect to Incomplete Decoding](https://mlsanity.com/paper/2002.02492/) (2020)
- [Controllable Neural Story Plot Generation via Reward Shaping](https://mlsanity.com/paper/1809.10736/) (2018)
- [Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence](https://mlsanity.com/paper/2105.08963/) (2021)
- [PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation](https://mlsanity.com/paper/2203.09100/) (2022)
- [Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoning](https://mlsanity.com/paper/2105.01311/) (2021)
- [Automatic Story Generation: Challenges and Attempts](https://mlsanity.com/paper/2102.12634/) (2021)
- [proScript: Partially Ordered Scripts Generation via Pre-trained Language Models](https://mlsanity.com/paper/2104.08251/) (2021)
- [Narrative Interpolation for Generating and Understanding Stories](https://mlsanity.com/paper/2008.07466/) (2020)
- [Plug-and-Blend: A Framework for Controllable Story Generation with Blended Control Codes](https://mlsanity.com/paper/2104.04039/) (2021)
- [GraphPlan: Story Generation by Planning with Event Graph](https://mlsanity.com/paper/2102.02977/) (2021)
- [EtriCA: Event-Triggered Context-Aware Story Generation Augmented by Cross Attention](https://mlsanity.com/paper/2210.12463/) (2022)

## Built on by (most cited first, 12 of 12)

- [What Makes a Good Story and How Can We Measure It? A Comprehensive Survey of Story Evaluation](https://mlsanity.com/paper/2408.14622/) (2024)
- [A Character-Centric Creative Story Generation via Imagination](https://mlsanity.com/paper/2409.16667/) (2024)
- [Avoidance Decoding for Diverse Multi-Branch Story Generation](https://mlsanity.com/paper/2509.02170/) (2025)
- [GPT-Powered Elicitation Interview Script Generator for Requirements Engineering Training](https://mlsanity.com/paper/2406.11439/) (2024)
- [MultiActor-Audiobook: Zero-Shot Audiobook Generation with Faces and Voices of Multiple Speakers](https://mlsanity.com/paper/2505.13082/) (2025)
- [Explaining Mixtures of Sources in News Articles](https://mlsanity.com/paper/2411.05192/) (2024)
- [FlexMUSE: Multimodal Unification and Semantics Enhancement Framework with Flexible interaction for Creative Writing](https://mlsanity.com/paper/2508.16230/) (2025)
- [A Universal Avoidance Method for Diverse Multi-branch Generation](https://mlsanity.com/paper/2604.17323/) (2026)
- [CreAgentive: An Agent Workflow Driven Multi-Category Creative Generation Engine](https://mlsanity.com/paper/2509.26461/) (2025)
- [LitVISTA: A Benchmark for Narrative Orchestration in Literary Text](https://mlsanity.com/paper/2601.06445/) (2026)
- [ConWriter: Transition-Constrained Stateful Long-Form Story Generation with Lightweight Neuro-Symbolic Consistency Control](https://mlsanity.com/paper/2608.05169/) (2026)
- [StorySpark: Module-wise Evolutionary Search for Story Premise Generation](https://mlsanity.com/paper/2608.12336/) (2026)

## Abstract (arXiv)

A human author can write any length of story without losing coherence. Also, they always bring the story to a proper ending, an ability that current language models lack. In this work, we present the LongStory for coherent, complete, and length-controlled long story generation. LongStory introduces two novel methodologies: (1) the long and short-term contexts weight calibrator (CWC) and (2) long story structural positions (LSP). The CWC adjusts weights for long-term context Memory and short-term context Cheating, acknowledging their distinct roles. The LSP employs discourse tokens to convey the structural positions of a long story. Trained on three datasets with varied average story lengths, LongStory outperforms other baselines, including the strong story generator Plotmachine, in coherence, completeness, relevance, and repetitiveness. We also perform zero-shot tests on each dataset to assess the model's ability to predict outcomes beyond its training data and validate our methodology by comparing its performance with variants of our model.