# Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models

arXiv: https://arxiv.org/abs/2309.02772 · HTML: https://mlsanity.com/paper/2309.02772/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 6 Sep 2023.

## Builds on (most cited first, 13 of 13)

- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X](https://mlsanity.com/paper/2303.17568/) (2023)
- [Beam Search Strategies for Neural Machine Translation](https://mlsanity.com/paper/1702.01806/) (2017)
- [Structured Chain-of-Thought Prompting for Code Generation](https://mlsanity.com/paper/2305.06599/) (2023)
- [Do Massively Pretrained Language Models Make Better Storytellers?](https://mlsanity.com/paper/1909.10705/) (2019)
- [SkCoder: A Sketch-based Approach for Automatic Code Generation](https://mlsanity.com/paper/2302.06144/) (2023)
- [AceCoder: Utilizing Existing Code to Enhance Code Generation](https://mlsanity.com/paper/2303.17780/) (2023)
- [On End-to-End Program Generation from User Intention by Deep Neural Networks](https://mlsanity.com/paper/1510.07211/) (2015)
- [Improving Diversity of Neural Text Generation via Inverse Probability Weighting](https://mlsanity.com/paper/2103.07649/) (2021)

## Built on by (most cited first, 30 of 38)

- [Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling](https://mlsanity.com/paper/2507.23370/) (2025)
- [Robustifying Safety-Aligned Large Language Models through Clean Data Curation](https://mlsanity.com/paper/2405.19358/) (2024)
- [Exploring the Impact of Temperature on Large Language Models:Hot or Cold?](https://mlsanity.com/paper/2506.07295/) (2025)
- [Look Inward to Explore Outward: Learning Temperature Policy from LLM Internal States via Hierarchical RL](https://mlsanity.com/paper/2602.13035/) (2026)
- [Proof Automation with Large Language Models](https://mlsanity.com/paper/2409.14274/) (2024)
- [What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond](https://mlsanity.com/paper/2503.20589/) (2025)
- [Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models](https://mlsanity.com/paper/2407.00456/) (2024)
- [Beyond Exponential Decay: Rethinking Error Accumulation in Large Language Models](https://mlsanity.com/paper/2505.24187/) (2025)
- [Klear-CodeTest: Scalable Test Case Generation for Code Reinforcement Learning](https://mlsanity.com/paper/2508.05710/) (2025)
- [MS-BART: Unified Modeling of Mass Spectra and Molecules for Structure Elucidation](https://mlsanity.com/paper/2510.20615/) (2025)
- [CyLens: Towards Reinventing Cyber Threat Intelligence in the Paradigm of Agentic Large Language Models](https://mlsanity.com/paper/2502.20791/) (2025)
- [The Consistency Hypothesis in Uncertainty Quantification for Large Language Models](https://mlsanity.com/paper/2506.21849/) (2025)
- [DecoRTL: A Run-time Decoding Framework for RTL Code Generation with LLMs](https://mlsanity.com/paper/2507.02226/) (2025)
- [Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs](https://mlsanity.com/paper/2510.05987/) (2025)
- [TreeCoder: Systematic Exploration and Optimisation of Decoding and Constraints for LLM Code Generation](https://mlsanity.com/paper/2511.22277/) (2025)
- [On LLMs' Internal Representation of Code Correctness](https://mlsanity.com/paper/2512.07404/) (2025)
- [What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworks](https://mlsanity.com/paper/2407.03007/) (2024)
- [Entropy Controllable Direct Preference Optimization](https://mlsanity.com/paper/2411.07595/) (2024)
- [From Token to Line: Enhancing Code Generation with a Long-Term Perspective](https://mlsanity.com/paper/2504.07433/) (2025)
- [Restoring Exploration after Post-Training: Latent Exploration Decoding for Large Reasoning Models](https://mlsanity.com/paper/2602.01698/) (2026)
- [SCALE: Self-uncertainty Conditioned Adaptive Looking and Execution for Vision-Language-Action Models](https://mlsanity.com/paper/2602.04208/) (2026)
- [LLMs Prompted for Graphs: Hallucinations and Generative Capabilities](https://mlsanity.com/paper/2409.00159/) (2024)
- [Improving Medical Diagnostics with Vision-Language Models: Convex Hull-Based Uncertainty Analysis](https://mlsanity.com/paper/2412.00056/) (2024)
- [From Recall to Reasoning: Automated Question Generation for Deeper Math Learning through Large Language Models](https://mlsanity.com/paper/2505.11899/) (2025)
- [Automated Extraction of Material Properties using LLM-based AI Agents](https://mlsanity.com/paper/2510.01235/) (2025)
- [ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generation](https://mlsanity.com/paper/2601.09703/) (2026)
- [Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Models](https://mlsanity.com/paper/2601.22629/) (2026)
- [Context-Enhanced Contrastive Search for Improved LLM Text Generation](https://mlsanity.com/paper/2504.21020/) (2025)
- [On Next-Token Prediction in LLMs: How End Goals Determine the Consistency of Decoding Algorithms](https://mlsanity.com/paper/2505.11183/) (2025)
- [Confidence-Modulated Speculative Decoding for Large Language Models](https://mlsanity.com/paper/2508.15371/) (2025)

## Abstract (arXiv)

Recently, Large Language Models (LLMs) have shown impressive abilities in code generation. However, existing LLMs' decoding strategies are designed for Natural Language (NL) generation, overlooking the differences between NL and programming languages (PL). Due to this oversight, a better decoding strategy for code generation remains an open question. In this paper, we conduct the first systematic study to explore a decoding strategy specialized in code generation. With an analysis of loss distributions of code tokens, we find that code tokens can be divided into two categories: challenging tokens that are difficult to predict and confident tokens that can be easily inferred. Among them, the challenging tokens mainly appear at the beginning of a code block. Inspired by the above findings, we propose a simple yet effective method: Adaptive Temperature (AdapT) sampling, which dynamically adjusts the temperature coefficient when decoding different tokens. We apply a larger temperature when sampling for challenging tokens, allowing LLMs to explore diverse choices. We employ a smaller temperature for confident tokens avoiding the influence of tail randomness noises. We apply AdapT sampling to LLMs with different sizes and conduct evaluations on two popular datasets. Results show that AdapT sampling significantly outperforms state-of-the-art decoding strategy.