LLMs Exhibit Significantly Lower Uncertainty in Creative Writing Than Professional Writers
Not on the map. Why?
Abstract (arXiv)
We argue that uncertainty is a key and understudied limitation of LLMs' performance in creative writing, which is often characterized as trite and clich'e-ridden. Literary theory identifies uncertainty as a necessary condition for creative expression, while current alignment strategies steer models away from uncertain outputs to ensure factuality and reduce hallucination. We formalize this tension by quantifying the ``uncertainty gap'' between human-authored stories and model-generated continuations. Through a controlled information-theoretic analysis of 28 LLMs on high-quality storytelling datasets, we demonstrate that human writing consistently exhibits significantly higher uncertainty than model outputs. We find that instruction-tuned and reasoning models exacerbate this trend compared to their base counterparts; furthermore, the gap is more pronounced in creative writing than in functional domains, and shows a consistent correlation with writing quality. Achieving human-level creativity requires new uncertainty-aware alignment paradigms that can distinguish between destructive hallucinations and the constructive ambiguity required for literary richness.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 49 of its 81 references that are in the map. First posted on arXiv 18 Feb 2026.
Builds on
49 of 81 references.
References · 49
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
- 2025Why Language Models Hallucinatecited by 205
- 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
- 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
- 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
- 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 168
- 2023Generative artificial intelligence enhances creativity but reduces the diversity of novel contentcited by 165
- 2023Conformal Language Modelingcited by 163
- 2024Homogenization Effects of Large Language Models on Human Creative Ideationcited by 127
- 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
37 more
- 2022Locally Typical Samplingcited by 108
- 2024Taming Overconfidence in LLMs: Reward Calibration in RLHFcited by 105
- 2025Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversitycited by 100
- 2023Ghostbuster: Detecting Text Ghostwritten by Large Language Modelscited by 90
- 2024Are Large Language Models Capable of Generating Human-Level Narratives?cited by 78
- 2022Creative Writing with an AI-Powered Writing Assistant: Perspectives from Professional Writerscited by 69
- 2024Preserving Diversity in Supervised Fine-Tuning of Large Language Modelscited by 68
- 2022Mutual Information Alleviates Hallucinations in Abstractive Summarizationcited by 67
- 2024Agents' Room: Narrative Generation through Multi-step Collaborationcited by 57
- 2024Aligning Language Models to Explicitly Handle Ambiguitycited by 55
- 2024Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Editscited by 49
- 2025Base Models Beat Aligned Models at Randomness and Creativitycited by 48
- 2024Echoes in AI: Quantifying lack of plot diversity in LLM outputscited by 47
- 2024Learning to Ask: When LLM Agents Meet Unclear Instructioncited by 38
- 2024Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Modelscited by 37
- 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
- 2025Learning to Reason for Long-Form Story Generationcited by 34
- 2024Creativity Has Left the Chat: The Price of Debiasing Language Modelscited by 29
- 2024Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generationcited by 27
- 2025Diverse Preference Learning for Capabilities and Alignmentcited by 25
- 2025AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computationcited by 24
- 2024Evaluating Creative Short Story Generation in Humans and Large Language Modelscited by 21
- 2025CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answeringcited by 15
- 2020Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representationcited by 13
- 2025Ask a Strong LLM Judge when Your Reward Model is Uncertaincited by 13
- 2025Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsingcited by 12
- 2025Rethinking Creativity Evaluation: A Critical Analysis of Existing Creativity Evaluationscited by 12
- 2025Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedbackcited by 10
- 2025TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Storiescited by 10
- 2024The Dearth of the Author in AI-Supported Writingcited by 9
- 2025Group-Aware Reinforcement Learning for Output Diversity in Large Language Modelscited by 9
- 2025Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanationcited by 8
- 2025Frankentext: Stitching random text fragments into long-form narrativescited by 8
- 2023FACE: Evaluating Natural Language Generation with Fourier Analysis of Cross-Entropycited by 6
- 2025The Geometry of Creative Variability: How Credal Sets Expose Calibration Gaps in Language Modelscited by 4
- 2025Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimizationcited by 2
- 2025Correct-Detect: Balancing Performance and Ambiguity Through the Lens of Coreference Resolution in LLMscited by 1
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Spoiler Alert: Narrative Forecasting as a Metric for Tension in LLM Storytellingcited by 1
- 2026Narrative Flattening: How Post-Training Compresses Thematic, Affective, and Stylistic Variation in LLM Fictioncited by 1
- 2026Magic, Madness, Heaven, Sin: LLM Output Diversity is Everything, Everywhere, All at Oncecited by 0
- 2026Verbalized and Internal Probabilities Are Coupled in Large Language Modelscited by 0
- 2026Verbalized and Internal Probabilities Are Coupled in Large Language Models30 Sep 2026
- 2026Narrative Flattening: How Post-Training Compresses Thematic, Affective, and Stylistic Variation in LLM Fiction27 May 2026
- 2026Spoiler Alert: Narrative Forecasting as a Metric for Tension in LLM Storytelling10 Apr 2026
- 2026Magic, Madness, Heaven, Sin: LLM Output Diversity is Everything, Everywhere, All at Once2 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026Lighting Up or Dimming Down? Exploring Dark Patterns of LLMs in Co-Creativity
- 2026The Storyteller in the Model: Narrative Pattern Inheritance, Escalation Dynamics, and Alignment Governance in LLMs
- 2026Towards Human-Level Book-Writing Capability
- 2026Directional Alignment and Narrative Agency in Human-LLM Co-Writing
- 2026Calibrated Surprise: An Information-Theoretic Account of Creative Quality
- 2026Human-Alignment, Calibration, and Activation Patterns in Large Language Model Uncertainty
- 2026QUIET: A Multi-Blank Cascaded Story Cloze Benchmark for LLM Creative Generation Capability
- 2025LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writing