# Before and After Temperature: A Distributional View of Creative LLM Generation

arXiv: https://arxiv.org/abs/2606.01451 · HTML: https://mlsanity.com/paper/2606.01451/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 31 May 2026.

## Builds on (most cited first, 20 of 20)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [Locally Typical Sampling](https://mlsanity.com/paper/2202.00666/) (2022)
- [Putting GPT-3's Creativity to the (Alternative Uses) Test](https://mlsanity.com/paper/2206.08932/) (2022)
- [Confidence, Not Perplexity: A Better Metric for the Creative Era of LLMs](https://mlsanity.com/paper/2510.08596/) (2025)

## Abstract (arXiv)

Reference-free evaluation of large language model (LLM) creativity relies on perplexity, entropy, and top-1 margin. We show that a much stronger signal lives one step earlier in the pipeline: in how sampling temperature reshapes the model's token distribution before the next token is drawn. On Llama-3.1-8B-Instruct generations of 500 open-ended creative prompts at T ∈ 0.3, 0.8, 1.5, a single per-token feature derived from this reshaping predicts the within-prompt creativity rank at Spearman ρ=0.918 against an averaged gpt-4o,/,gemini-2.5-pro judge (n=500) and ρ=0.870 against a three-rater human-majority ranking (n=150). Each of four standard reference-free baselines (self-perplexity, mean predictive entropy, top-1 margin, gzip compression ratio) tops out at |ρ|!≈!0.76 on both ground truths: a gap of +0.165 on averaged-LLM and +0.110 on human-majority, both far larger than the spread among the baselines themselves. The two ground-truth panels agree with each other at ρ=0.83, above the inter-human ceiling of ρ=0.77, so the comparison is not bottlenecked by judge noise. Mechanistically, the win comes from a sharp distributional signature of the incoherence regime: at T=1.5 the cumulative-mass width n95(q) inflates from ~!1 to ~!131 tokens and post-temperature mass leaks off the pre-temperature top-90% plausible set by about 13 percentage points. The per-token aggregates do not separate T=0.8 from T=0.3; discriminating the two coherent regimes is left to sequence-level features.