mlsanityData to 7 Oct 2026

Towards Human-Level Book-Writing Capability

arXiv 2605.17064 ↗First posted 16 May 2026cs.AIcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Large language models are optimized for instruction following and agentic tasks remain poorly aligned with the requirements of high-quality creative writing. We show that a purpose-built creative writing model can outperform both GPT-5.5 and Claude Opus 4.8 on writing quality evaluation. Fiction frequently depends on behaviors that assistant-tuned models are explicitly trained to avoid, particularly deception, moral ambiguity, and unreliable narration. As a result, generated stories often appear structurally correct while remaining stylistically generic, overly explanatory, or weakly grounded in human literary behavior. We present a dataset construction and training framework for book-scale creative writing that reframes supervised fine-tuning as a prompt-to-book generation task grounded in human-authored fiction. Starting from public-domain novels, we derive a multi-resolution Planning Scaffold by summarizing each book at progressively finer levels, from a high-level premise to chapter- and scene-level structure. We then invert this hierarchy during training: the model learns to expand a prompt into increasingly detailed plans and finally into the original human-authored book text. This formulation preserves human prose as the final supervised target while using intermediate summaries to make book-scale generation learnable. We train a long-context language model on these prompt-to-book trajectories and show that this objective shifts generation away from assistant-style prose and toward human literary writing.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 22 of its 27 references that are in the map. First posted on arXiv 16 May 2026.

Builds on

22 of 27 references.

References · 22

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  6. 2018Hierarchical Neural Story Generationcited by 1,182
  7. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
  8. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  9. 2025Muon is Scalable for LLM Trainingcited by 381
  10. 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 232
  11. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  12. 2019A Study of BFLOAT16 for Deep Learning Trainingcited by 227
10 more
  1. 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
  2. 2019Strategies for Structuring Story Generationcited by 124
  3. 2020PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Trackingcited by 104
  4. 2017Event Representations for Automated Story Generation with Deep Neural Netscited by 62
  5. 2014Narrative Planning: Balancing Plot and Charactercited by 61
  6. 2024Self-Boosting Large Language Models with Synthetic Preference Datacited by 40
  7. 2024Generating Long-form Story Using Dynamic Hierarchical Outlining with Memory-Enhancementcited by 30
  8. 2025Stochastic Rounding for LLM Training: Theory and Practicecited by 10
  9. 2024The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generationcited by 9
  10. 2025Parameterized Synthetic Text Generation with SimpleStoriescited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writing
  2. 2025Learning to Reason for Long-Form Story Generation
  3. 2024Agents' Room: Narrative Generation through Multi-step Collaboration
  4. 2026When Reasoning Supervision Hurts: TTCW-Based Long-Form Literary Review Generation
  5. 2026POLARIS: Guiding Small Models to Write Long Stories
  6. 2026Narrative Flattening: How Post-Training Compresses Thematic, Affective, and Stylistic Variation in LLM Fiction
  7. 2026LLMs Exhibit Significantly Lower Uncertainty in Creative Writing Than Professional Writers
  8. 2026Generating Constructive Feedback on Stories via Reinforcement Learning