# Top-H Decoding: Adapting the Creativity and Coherence with Bounded Entropy in Text Generation

arXiv: https://arxiv.org/abs/2509.02510 · HTML: https://mlsanity.com/paper/2509.02510/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Builds on (most cited first, 12 of 12)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs](https://mlsanity.com/paper/2407.01082/) (2024)
- [Truncation Sampling as Language Model Desmoothing](https://mlsanity.com/paper/2210.15191/) (2022)

## Built on by (most cited first, 9 of 9)

- [Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning](https://mlsanity.com/paper/2602.10273/) (2026)
- [LLM Review: Enhancing Creative Writing via Blind Peer Review Feedback](https://mlsanity.com/paper/2601.08003/) (2026)
- [From Filters to VLMs: Benchmarking Defogging Methods through Object Detection and Segmentation Performance](https://mlsanity.com/paper/2510.03906/) (2025)
- [Geometry-Aware Decoding with Wasserstein-Regularized Truncation and Mass Penalties for Large Language Models](https://mlsanity.com/paper/2602.10346/) (2026)
- [Video Reasoning without Training](https://mlsanity.com/paper/2510.17045/) (2025)
- [Sampling More, Getting Less: Calibration is the Diversity Bottleneck in LLMs](https://mlsanity.com/paper/2605.11128/) (2026)
- [Beyond Entropy: Learning from Token-Level Distributional Deviations for LLM Reasoning](https://mlsanity.com/paper/2606.19771/) (2026)
- [Entropy-informed Decoding: Adaptive Information-Driven Branching](https://mlsanity.com/paper/2605.09745/) (2026)
- [Truncation Samplers Are Nested: Comparing Decoding Rules Is a Measurement Problem](https://mlsanity.com/paper/2610.15570/) (2026)

## Abstract (arXiv)

Large language models (LLMs), despite their impressive performance across a wide range of tasks, often struggle to balance two competing objectives in open-ended text generation: fostering diversity and creativity while preserving logical coherence. Existing truncated sampling techniques, including temperature scaling, top-p (nucleus) sampling, and min-p sampling, aim to manage this trade-off. However, they exhibit limitations, particularly in the effective incorporation of the confidence of the model into the corresponding sampling strategy. For example, min-p sampling relies on a single top token as a heuristic for confidence, eventually underutilizing the information of the probability distribution. Toward effective incorporation of the confidence of the model, in this paper, we present **top-H** decoding. We first establish the theoretical foundation of the interplay between creativity and coherence in truncated sampling by formulating an **entropy-constrained minimum divergence** problem. We then prove this minimization problem to be equivalent to an **entropy-constrained mass maximization** (ECMM) problem, which is NP-hard. Finally, we present top-H decoding, a computationally efficient greedy algorithm to solve the ECMM problem. Extensive empirical evaluations demonstrate that top-H outperforms the state-of-the-art (SoTA) alternative of min-p sampling by up to **25.63%** on creative writing benchmarks, while maintaining robustness on question-answering datasets such as GPQA, GSM8K, and MT-Bench. Additionally, an *LLM-as-judge* evaluation confirms that top-H indeed produces coherent outputs even at higher temperatures, where creativity is especially critical. In summary, top-H advances SoTA in open-ended text generation and can be *easily integrated* into creative writing applications. The code is available at https://github.com/ErfanBaghaei/Top-H-Decoding.