mlsanityData to 7 Oct 2026

Learning to Reason for Long-Form Story Generation

arXiv 2503.22828 ↗First posted 28 Mar 2025cs.CLcited by 34

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Generating high-quality stories spanning thousands of tokens requires competency across a variety of skills, from tracking plot and character arcs to keeping a consistent and engaging style. Due to the difficulty of sourcing labeled datasets and precise quality measurements, most work using large language models (LLMs) for long-form story generation uses combinations of hand-designed prompting techniques to elicit author-like behavior. This is a manual process that is highly dependent on the specific story-generation task. Motivated by the recent success of applying RL with Verifiable Rewards to domains like math and coding, we propose a general story-generation task (Next-Chapter Prediction) and a reward formulation (Verified Rewards via Completion Likelihood Improvement) that allows us to use an unlabeled book dataset as a learning signal for reasoning. We learn to reason over a story's condensed information and generate a detailed plan for the next chapter. Our reasoning is evaluated via the chapters it helps a story-generator create, and compared against non-trained and supervised finetuning (SFT) baselines. Pairwise human judgments reveal the chapters our learned reasoning produces are preferred across almost all metrics, and the effect is more pronounced in Scifi and Fantasy genres.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 24 of its 63 references that are in the map. First posted on arXiv 28 Mar 2025.

Builds on

24 of 63 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Models28 Feb
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Models24 Feb

Other references · 22

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  6. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  7. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  8. 2025Process Reinforcement through Implicit Rewardscited by 478
  9. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  10. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 260
  11. 2023RecurrentGPT: Interactive Generation of (Arbitrarily) Long Textcited by 84
  12. 2024OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Frameworkcited by 70
10 more
  1. 2024Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewardingcited by 53
  2. 2024LongGenBench: Benchmarking Long-Form Generation in Long Context LLMscited by 47
  3. 2024HelloBench: Evaluating Long Text Generation Capabilities of Large Language Modelscited by 42
  4. 2024Character is Destiny: Can Role-Playing Language Agents Make Persona-Driven Decisions?cited by 34
  5. 2025Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Datacited by 29
  6. 2025LADDER: Self-Improving LLMs Through Recursive Problem Decompositioncited by 24
  7. 2022Open-world Story Generation with Structured Knowledge Enhancement: A Comprehensive Surveycited by 21
  8. 2024Evaluating Creative Short Story Generation in Humans and Large Language Modelscited by 21
  9. 2025CLIPPER: Compression enables long-context synthetic data generationcited by 9
  10. 2024LASeR: Learning to Adaptively Select Reward Models with Multi-Armed Banditscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 22.

34 papers.

Sort:

By area:

  1. 2025CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learningcited by 36
  2. 2026TIPS: Turn-Level Information-Potential Reward Shaping for Search-Augmented LLMscited by 24
  3. 2025Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Densecited by 19
  4. 2025BLEUBERI: BLEU is a surprisingly effective reward for instruction followingcited by 17
  5. 2025NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learningcited by 15
  6. 2025Scalpel vs. Hammer: GRPO Amplifies Existing Capabilities, SFT Replaces Themcited by 14
  7. 2025SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimizationcited by 10
  8. 2025Bootstrapping Task Spaces for Self-Improvementcited by 10
  9. 2026Learning to Simulate Human Dialoguecited by 9
  10. 2025Can Large Language Models Develop Strategic Reasoning? Post-training Insights from Learning Chesscited by 8
  11. 2025Escaping the Verifier: Learning to Reason via Demonstrationscited by 7
  12. 2025Plantain: Plan-Answer Interleaved Reasoningcited by 5
22 more
  1. 2026LLMs Exhibit Significantly Lower Uncertainty in Creative Writing Than Professional Writerscited by 4
  2. 2026Deriving Character Logic from Storyline as Codified Decision Treescited by 3
  3. 2026CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioningcited by 3
  4. 2026CoMAI: A Collaborative Multi-Agent Framework for Robust and Equitable Interview Evaluationcited by 2
  5. 2026Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluationcited by 2
  6. 2026AI as a Tool for Simulation-Based Experiments in Literary Studiescited by 2
  7. 2026Right in the Right Way: LM Training with Verifiable Rewards and Human Demonstrationscited by 2
  8. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 2
  9. 2025Aether Weaver: Multimodal Affective Narrative Co-Generation with Dynamic Scene Graphscited by 1
  10. 2025SciTrek: Evaluating and Improving Long-Context Numerical Reasoning over Scientific Articlescited by 1
  11. 2026Spoiler Alert: Narrative Forecasting as a Metric for Tension in LLM Storytellingcited by 1
  12. 2026Self-Guard: Defending Large Reasoning Models via enhanced self-reflectioncited by 0
  13. 2026PlotTwist: A Creative Plot Generation Framework with Small Language Modelscited by 0
  14. 2026MoRI: Learning Motivation-Grounded Reasoning for Scientific Ideation in Large Language Modelscited by 0
  15. 2026LLMs Should Incorporate Explicit Mechanisms for Human Empathycited by 0
  16. 2026Think Before you Write: QA-Guided Reasoning for Character Descriptions in Bookscited by 0
  17. 2026NARRA-Gym for Evaluating Interactive Narrative Agentscited by 0
  18. 2026Likelihood scoring for continuations of mathematical text: a self-supervised benchmark with tests for shortcut vulnerabilitiescited by 0
  19. 2026BOOKMARKS: Efficient Active Storyline Memory for Role-playingcited by 0
  20. 2026POLARIS: Guiding Small Models to Write Long Storiescited by 0
  21. 2026SoftmaxGRPO: Learning to Reason using Softmax Advantage Group Estimationcited by 0
  22. 2026Strategically Diverse Sampling for Self-Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026StoryAlign: Evaluating and Training Reward Models for Story Generation
  2. 2025EvolvR: Self-Evolving Pairwise Reasoning for Story Evaluation to Enhance Generation
  3. 2023GROVE: A Retrieval-augmented Complex Story Generation Framework with A Forest of Evidence
  4. 2025Language Models Might Not Understand You: Evaluating Theory of Mind via Story Prompting
  5. 2026StoryCoder: Narrative Reformulation for Structured Reasoning in LLM Code Generation
  6. 2026Towards Human-Level Book-Writing Capability
  7. 2025StoryWriter: A Multi-Agent Framework for Long Story Generation
  8. 2026When Reasoning Supervision Hurts: TTCW-Based Long-Form Literary Review Generation