mlsanityData to 7 Oct 2026

LLMs Exhibit Significantly Lower Uncertainty in Creative Writing Than Professional Writers

arXiv 2602.16162 ↗First posted 18 Feb 2026cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

We argue that uncertainty is a key and understudied limitation of LLMs' performance in creative writing, which is often characterized as trite and clich'e-ridden. Literary theory identifies uncertainty as a necessary condition for creative expression, while current alignment strategies steer models away from uncertain outputs to ensure factuality and reduce hallucination. We formalize this tension by quantifying the ``uncertainty gap'' between human-authored stories and model-generated continuations. Through a controlled information-theoretic analysis of 28 LLMs on high-quality storytelling datasets, we demonstrate that human writing consistently exhibits significantly higher uncertainty than model outputs. We find that instruction-tuned and reasoning models exacerbate this trend compared to their base counterparts; furthermore, the gap is more pronounced in creative writing than in functional domains, and shows a consistent correlation with writing quality. Achieving human-level creativity requires new uncertainty-aware alignment paradigms that can distinguish between destructive hallucinations and the constructive ambiguity required for literary richness.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 49 of its 81 references that are in the map. First posted on arXiv 18 Feb 2026.

Builds on

49 of 81 references.

References · 49

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  3. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
  4. 2025Why Language Models Hallucinatecited by 205
  5. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  6. 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
  7. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  8. 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 168
  9. 2023Generative artificial intelligence enhances creativity but reduces the diversity of novel contentcited by 165
  10. 2023Conformal Language Modelingcited by 163
  11. 2024Homogenization Effects of Large Language Models on Human Creative Ideationcited by 127
  12. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
37 more
  1. 2022Locally Typical Samplingcited by 108
  2. 2024Taming Overconfidence in LLMs: Reward Calibration in RLHFcited by 105
  3. 2025Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversitycited by 100
  4. 2023Ghostbuster: Detecting Text Ghostwritten by Large Language Modelscited by 90
  5. 2024Are Large Language Models Capable of Generating Human-Level Narratives?cited by 78
  6. 2022Creative Writing with an AI-Powered Writing Assistant: Perspectives from Professional Writerscited by 69
  7. 2024Preserving Diversity in Supervised Fine-Tuning of Large Language Modelscited by 68
  8. 2022Mutual Information Alleviates Hallucinations in Abstractive Summarizationcited by 67
  9. 2024Agents' Room: Narrative Generation through Multi-step Collaborationcited by 57
  10. 2024Aligning Language Models to Explicitly Handle Ambiguitycited by 55
  11. 2024Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Editscited by 49
  12. 2025Base Models Beat Aligned Models at Randomness and Creativitycited by 48
  13. 2024Echoes in AI: Quantifying lack of plot diversity in LLM outputscited by 47
  14. 2024Learning to Ask: When LLM Agents Meet Unclear Instructioncited by 38
  15. 2024Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Modelscited by 37
  16. 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
  17. 2025Learning to Reason for Long-Form Story Generationcited by 34
  18. 2024Creativity Has Left the Chat: The Price of Debiasing Language Modelscited by 29
  19. 2024Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generationcited by 27
  20. 2025Diverse Preference Learning for Capabilities and Alignmentcited by 25
  21. 2025AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computationcited by 24
  22. 2024Evaluating Creative Short Story Generation in Humans and Large Language Modelscited by 21
  23. 2025CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answeringcited by 15
  24. 2020Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representationcited by 13
  25. 2025Ask a Strong LLM Judge when Your Reward Model is Uncertaincited by 13
  26. 2025Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsingcited by 12
  27. 2025Rethinking Creativity Evaluation: A Critical Analysis of Existing Creativity Evaluationscited by 12
  28. 2025Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedbackcited by 10
  29. 2025TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Storiescited by 10
  30. 2024The Dearth of the Author in AI-Supported Writingcited by 9
  31. 2025Group-Aware Reinforcement Learning for Output Diversity in Large Language Modelscited by 9
  32. 2025Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanationcited by 8
  33. 2025Frankentext: Stitching random text fragments into long-form narrativescited by 8
  34. 2023FACE: Evaluating Natural Language Generation with Fourier Analysis of Cross-Entropycited by 6
  35. 2025The Geometry of Creative Variability: How Credal Sets Expose Calibration Gaps in Language Modelscited by 4
  36. 2025Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimizationcited by 2
  37. 2025Correct-Detect: Balancing Performance and Ambiguity Through the Lens of Coreference Resolution in LLMscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Lighting Up or Dimming Down? Exploring Dark Patterns of LLMs in Co-Creativity
  2. 2026The Storyteller in the Model: Narrative Pattern Inheritance, Escalation Dynamics, and Alignment Governance in LLMs
  3. 2026Towards Human-Level Book-Writing Capability
  4. 2026Directional Alignment and Narrative Agency in Human-LLM Co-Writing
  5. 2026Calibrated Surprise: An Information-Theoretic Account of Creative Quality
  6. 2026Human-Alignment, Calibration, and Activation Patterns in Large Language Model Uncertainty
  7. 2026QUIET: A Multi-Blank Cascaded Story Cloze Benchmark for LLM Creative Generation Capability
  8. 2025LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writing