# Integrating Planning into Single-Turn Long-Form Text Generation

arXiv: https://arxiv.org/abs/2410.06203 · HTML: https://mlsanity.com/paper/2410.06203/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 28 of 28)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [Generating Wikipedia by Summarizing Long Sequences](https://mlsanity.com/paper/1801.10198/) (2018)
- [Building Cooperative Embodied Agents Modularly with Large Language Models](https://mlsanity.com/paper/2307.02485/) (2023)
- [Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval](https://mlsanity.com/paper/2108.05540/) (2021)
- [Re3: Generating Longer Stories With Recursive Reprompting and Revision](https://mlsanity.com/paper/2210.06774/) (2022)
- [ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models](https://mlsanity.com/paper/2305.18323/) (2023)
- [Hurdles to Progress in Long-form Question Answering](https://mlsanity.com/paper/2103.06332/) (2021)
- [Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models](https://mlsanity.com/paper/2402.14207/) (2024)
- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [A Critical Evaluation of Evaluations for Long-form Question Answering](https://mlsanity.com/paper/2305.18201/) (2023)
- [Discourse-Aware Neural Extractive Text Summarization](https://mlsanity.com/paper/1910.14142/) (2019)
- [Dynamic Planning with a LLM](https://mlsanity.com/paper/2308.06391/) (2023)
- [Read before Generate! Faithful Long Form Question Answering with Machine Reading](https://mlsanity.com/paper/2203.00343/) (2022)
- [Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Sampling](https://mlsanity.com/paper/2305.09993/) (2023)
- [ChapterBreak: A Challenge Dataset for Long-Range Language Models](https://mlsanity.com/paper/2204.10878/) (2022)
- ["Why is this misleading?": Detecting News Headline Hallucinations with Explanations](https://mlsanity.com/paper/2302.05852/) (2023)
- [SciNews: From Scholarly Complexities to Public Narratives -- A Dataset for Scientific News Report Generation](https://mlsanity.com/paper/2403.17768/) (2024)
- [Expository Text Generation: Imitate, Retrieve, Paraphrase](https://mlsanity.com/paper/2305.03276/) (2023)

## Built on by (most cited first, 7 of 7)

- [OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking](https://mlsanity.com/paper/2501.09751/) (2025)
- [Can LLMs Automate Fact-Checking Article Writing?](https://mlsanity.com/paper/2503.17684/) (2025)
- [WikiAutoGen: Towards Multi-Modal Wikipedia-Style Article Generation](https://mlsanity.com/paper/2503.19065/) (2025)
- [Disco-RAG: Discourse-Aware Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.04377/) (2026)
- [Story2Proposal: A Scaffold for Structured Scientific Paper Writing](https://mlsanity.com/paper/2603.27065/) (2026)
- [IS-CoT: Breaking the Long-form Generation Collapse via Interleaved Structural Thinking](https://mlsanity.com/paper/2606.09709/) (2026)
- [AdaptAgent: A Multi-agent, Domain-Guided Reasoning Framework for Code Adaptation](https://mlsanity.com/paper/2608.04459/) (2026)

## Abstract (arXiv)

Generating high-quality, in-depth textual documents, such as academic papers, news articles, Wikipedia entries, and books, remains a significant challenge for Large Language Models (LLMs). In this paper, we propose to use planning to generate long form content. To achieve our goal, we generate intermediate steps via an auxiliary task that teaches the LLM to plan, reason and structure before generating the final text. Our main novelty lies in a single auxiliary task that does not require multiple rounds of prompting or planning. To overcome the scarcity of training data for these intermediate steps, we leverage LLMs to generate synthetic intermediate writing data such as outlines, key information and summaries from existing full articles. Our experiments demonstrate on two datasets from different domains, namely the scientific news dataset SciNews and Wikipedia datasets in KILT-Wiki and FreshWiki, that LLMs fine-tuned with the auxiliary task generate higher quality documents. We observed +2.5% improvement in ROUGE-Lsum, and a strong 3.60 overall win/loss ratio via human SxS evaluation, with clear wins in organization, relevance, and verifiability.