mlsanityData to 7 Oct 2026

The Effects of Incremental Instruction Delivery on Language-Model Creative Writing

arXiv 2609.33738 ↗First posted 27 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models are increasingly used as interactive writing tools, where users develop stories, revise ideas, and introduce new requirements across multiple turns rather than specifying a complete brief upfront. Yet most evidence on multi-turn instruction degradation comes from tasks with objectively verifiable outcomes, leaving unclear whether incremental interaction harms creative artifacts in ways that explicit requirement checks cannot capture. We study this question using 160 human-authored creative-writing tasks across six genres, presenting each intended specification either upfront or progressively over 5-9 turns to six distinct open-weight model families, yielding 960 matched pairs. Progressive delivery reduces explicit constraint adherence and produces its largest writing-quality degradation in structure/coherence. The structural gap persists among outputs with equal observed adherence, suggesting that measured requirement loss alone does not explain the observed structural difference. We define Creative Integrity as a compact measure of joint adherence and narrative structure; under incremental delivery, models retain 71.2% of FULL Creative Integrity (95% CI [68.2%, 74.3%]). A three-rater human study over 50 matched pairs independently recovers FULL advantages in structure/coherence, craft, and genre effectiveness, while automated scores remain positively associated with aggregated human ratings. These findings show that interactive creative-writing systems should be evaluated not only on whether requirements survive conversation, but also on whether evolving requirements remain coherently integrated into the final artifact. Our dataset, benchmarks, and source code are available at: https://github.com/solusops/SISTER-2026-Team19

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 19 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

16 of 19 references.

References · 16

  1. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  2. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  3. 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 232
  4. 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
  5. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  6. 2025WritingBench: A Comprehensive Benchmark for Generative Writingcited by 95
  7. 2024Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Followingcited by 94
  8. 2022Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generationcited by 73
  9. 2021Wordcraft: a Human-AI Collaborative Editor for Story Writingcited by 41
  10. 2025LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writingcited by 39
  11. 2024Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluationcited by 32
  12. 2025StructFlowBench: A Structured Flow Benchmark for Multi-turn Instruction Followingcited by 28
4 more
  1. 2024CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraintscited by 22
  2. 2025Rethinking Creativity Evaluation: A Critical Analysis of Existing Creativity Evaluationscited by 12
  3. 2025One Battle After Another: Probing LLMs' Limits on Multi-Turn Instruction Following with a Benchmark Evolving Frameworkcited by 9
  4. 2026SEQUOR: A Multi-Turn Benchmark for Realistic Constraint Followingcited by 5

Built on by

Not cited yet.