# EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling

arXiv: https://arxiv.org/abs/2403.14541 · HTML: https://mlsanity.com/paper/2403.14541/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Builds on (most cited first, 27 of 27)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [An Empirical Study of the Non-determinism of ChatGPT in Code Generation](https://mlsanity.com/paper/2308.02828/) (2023)
- [Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions](https://mlsanity.com/paper/2306.04140/) (2023)
- [LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimization](https://mlsanity.com/paper/2306.01102/) (2023)
- [Paraphrase Generation with Deep Reinforcement Learning](https://mlsanity.com/paper/1711.00279/) (2017)
- [Lost in Translation: Loss and Decay of Linguistic Richness in Machine Translation](https://mlsanity.com/paper/1906.12068/) (2019)
- [Long Document Summarization with Top-down and Bottom-up Inference](https://mlsanity.com/paper/2203.07586/) (2022)
- [mFACE: Multilingual Summarization with Factual Consistency Evaluation](https://mlsanity.com/paper/2212.10622/) (2022)
- [Show, Ask, Attend, and Answer: A Strong Baseline For Visual Question Answering](https://mlsanity.com/paper/1704.03162/) (2017)
- [SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering](https://mlsanity.com/paper/1812.03593/) (2018)
- [Bidirectional Attentive Memory Networks for Question Answering over Knowledge Bases](https://mlsanity.com/paper/1903.02188/) (2019)
- [Focus Attention: Promoting Faithfulness and Diversity in Summarization](https://mlsanity.com/paper/2105.11921/) (2021)
- [Token-level Adaptive Training for Neural Machine Translation](https://mlsanity.com/paper/2010.04380/) (2020)
- [Multi-style Generative Reading Comprehension](https://mlsanity.com/paper/1901.02262/) (2019)
- [Mixup Decoding for Diverse Machine Translation](https://mlsanity.com/paper/2109.03402/) (2021)
- [Community Question Answering Entity Linking via Leveraging Auxiliary Data](https://mlsanity.com/paper/2205.11917/) (2022)
- [Unsupervised Question Answering via Answer Diversifying](https://mlsanity.com/paper/2208.10813/) (2022)

## Built on by (most cited first, 30 of 45)

- [Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs](https://mlsanity.com/paper/2407.01082/) (2024)
- [Modifying Large Language Model Post-Training for Diverse Creative Writing](https://mlsanity.com/paper/2503.17126/) (2025)
- [Optimizing Temperature for Language Models with Multi-Sample Inference](https://mlsanity.com/paper/2502.05234/) (2025)
- [Reward Is Enough: LLMs Are In-Context Reinforcement Learners](https://mlsanity.com/paper/2506.06303/) (2025)
- [Look Inward to Explore Outward: Learning Temperature Policy from LLM Internal States via Hierarchical RL](https://mlsanity.com/paper/2602.13035/) (2026)
- [Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities](https://mlsanity.com/paper/2501.12147/) (2025)
- [Process-based Self-Rewarding Language Models](https://mlsanity.com/paper/2503.03746/) (2025)
- [On the Effect of Sampling Diversity in Scaling LLM Inference](https://mlsanity.com/paper/2502.11027/) (2025)
- [Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs](https://mlsanity.com/paper/2510.01218/) (2025)
- [G2: Guided Generation for Enhanced Output Diversity in LLMs](https://mlsanity.com/paper/2511.00432/) (2025)
- [Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners](https://mlsanity.com/paper/2405.13816/) (2024)
- [Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregation](https://mlsanity.com/paper/2502.19830/) (2025)
- [DiffusionGemma Technical Report](https://mlsanity.com/paper/2608.00146/) (2026)
- [Adaptive Decoding via Latent Preference Optimization](https://mlsanity.com/paper/2411.09661/) (2024)
- [Let it Calm: Exploratory Annealed Decoding for Verifiable Reinforcement Learning](https://mlsanity.com/paper/2510.05251/) (2025)
- [Exploring Multi-Temperature Strategies for Token- and Rollout-Level Control in RLVR](https://mlsanity.com/paper/2510.08892/) (2025)
- [Flaming-hot Initiation with Regular Execution Sampling for Large Language Models](https://mlsanity.com/paper/2410.21236/) (2024)
- [PATS: Process-Level Adaptive Thinking Mode Switching](https://mlsanity.com/paper/2505.19250/) (2025)
- [Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decoding](https://mlsanity.com/paper/2504.01281/) (2025)
- [Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs](https://mlsanity.com/paper/2510.05987/) (2025)
- [SCALE: Self-uncertainty Conditioned Adaptive Looking and Execution for Vision-Language-Action Models](https://mlsanity.com/paper/2602.04208/) (2026)
- [Efficient Reasoning with Balanced Thinking](https://mlsanity.com/paper/2603.12372/) (2026)
- [Think Outside the Policy: In-Context Steered Policy Optimization](https://mlsanity.com/paper/2510.26519/) (2025)
- [Entropy-Aware Token Rejection for Improving Speculative Decoding](https://mlsanity.com/paper/2512.23765/) (2025)
- [Learning Adaptive LLM Decoding](https://mlsanity.com/paper/2603.09065/) (2026)
- [From Confidence to Collapse in LLM Factual Robustness](https://mlsanity.com/paper/2508.16267/) (2025)
- [EDCO: Dynamic Curriculum Orchestration for Domain-specific Large Language Model Fine-tuning](https://mlsanity.com/paper/2601.03725/) (2026)
- [Certain Head, Uncertain Tail: Expert-Sample for Test-Time Scaling in Fine-Grained MoE](https://mlsanity.com/paper/2602.02443/) (2026)
- [AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoning](https://mlsanity.com/paper/2608.29988/) (2026)
- [T2 of Thoughts: Temperature Tree Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2405.14075/) (2024)

## Abstract (arXiv)

Recently, Large Language Models (LLMs) have demonstrated outstanding performance across a wide range of downstream language tasks. Temperature sampling is a commonly used decoding strategy for LLMs' generation process. However, a fixed temperature parameter is used in most cases, which may not always be an optimal choice for balancing generation quality and diversity. In this paper, we propose an effective Entropy-based Dynamic Temperature (EDT) Sampling method, to achieve a more balanced performance in terms of both generation quality and diversity by dynamically selecting the temperature parameter. Additionally, we also show model performance and comprehensive analyses for 4 different generation benchmarks. Our experiments show that EDT significantly outperforms the existing strategies across different tasks.