# Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries

arXiv: https://arxiv.org/abs/2010.07074 · HTML: https://mlsanity.com/paper/2010.07074/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2020.

## Built on by (most cited first, 19 of 19)

- [Large Concept Models: Language Modeling in a Sentence Representation Space](https://mlsanity.com/paper/2412.08821/) (2024)
- [RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text](https://mlsanity.com/paper/2305.13304/) (2023)
- [DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization](https://mlsanity.com/paper/2110.08168/) (2021)
- [Weaver: Foundation Models for Creative Writing](https://mlsanity.com/paper/2401.17268/) (2024)
- [Language Models can Self-Lengthen to Generate Long Texts](https://mlsanity.com/paper/2410.23933/) (2024)
- [Navigating the Path of Writing: Outline-guided Text Generation with Large Language Models](https://mlsanity.com/paper/2404.13919/) (2024)
- [MoPS: Modular Story Premise Synthesis for Open-Ended Automatic Story Generation](https://mlsanity.com/paper/2406.05690/) (2024)
- [Ex3: Automatic Novel Writing by Extracting, Excelsior and Expanding](https://mlsanity.com/paper/2408.08506/) (2024)
- [Pap2Pat: Benchmarking Outline-Guided Long-Text Patent Generation with Patent-Paper Pairs](https://mlsanity.com/paper/2410.07009/) (2024)
- [CoRRPUS: Code-based Structured Prompting for Neurosymbolic Story Understanding](https://mlsanity.com/paper/2212.10754/) (2022)
- [End-to-end Story Plot Generator](https://mlsanity.com/paper/2310.08796/) (2023)
- [Advancing Precise Outline-Conditioned Text Generation with Task Duality and Explicit Outline Control](https://mlsanity.com/paper/2305.14459/) (2023)
- [Who Wrote the Book? Detecting and Attributing LLM Ghostwriters](https://mlsanity.com/paper/2603.28054/) (2026)
- [Goal-driven text descriptions for images](https://mlsanity.com/paper/2108.12575/) (2021)
- [Plot Writing From Pre-Trained Language Models](https://mlsanity.com/paper/2206.03021/) (2022)
- [Learning to Explore and Select for Coverage-Conditioned Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.01158/) (2024)
- [Long text outline generation: Chinese text outline based on unsupervised framework and large language mode](https://mlsanity.com/paper/2412.00810/) (2024)
- [Writing Like the Best: Exemplar-Based Expository Text Generation](https://mlsanity.com/paper/2505.18859/) (2025)
- [MultiGhostBench: A Multilingual Benchmark for Long-Form LLM-Generated Text Attribution under Distribution Shifts](https://mlsanity.com/paper/2609.02379/) (2026)

## Abstract (arXiv)

The difficulty of generating coherent long texts lies in the fact that existing models overwhelmingly focus on predicting local words, and cannot make high level plans on what to generate or capture the high-level discourse dependencies between chunks of texts. Inspired by human writing processes, where a list of bullet points or a catalog is first outlined, and then each bullet point is expanded to form the whole article, we propose it SOE, a pipelined system that involves of summarizing, outlining and elaborating for long text generation: the model first outlines the summaries for different segments of long texts, and then elaborates on each bullet point to generate the corresponding segment. To avoid the labor-intensive process of summary soliciting, we propose the it reconstruction strategy, which extracts segment summaries in an unsupervised manner by selecting its most informative part to reconstruct the segment. The proposed generation system comes with the following merits: (1) the summary provides high-level guidance for text generation and avoids the local minimum of individual word predictions; (2) the high-level discourse dependencies are captured in the conditional dependencies between summaries and are preserved during the summary expansion process and (3) additionally, we are able to consider significantly more contexts by representing contexts as concise summaries. Extensive experiments demonstrate that SOE produces long texts with significantly better quality, along with faster convergence speed.