mlsanityData to 7 Oct 2026

Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models

arXiv 2309.02772 ↗First posted 6 Sep 2023cs.SE cs.CLcited by 37

Not on the map. Why?

Abstract (arXiv)

Recently, Large Language Models (LLMs) have shown impressive abilities in code generation. However, existing LLMs' decoding strategies are designed for Natural Language (NL) generation, overlooking the differences between NL and programming languages (PL). Due to this oversight, a better decoding strategy for code generation remains an open question. In this paper, we conduct the first systematic study to explore a decoding strategy specialized in code generation. With an analysis of loss distributions of code tokens, we find that code tokens can be divided into two categories: challenging tokens that are difficult to predict and confident tokens that can be easily inferred. Among them, the challenging tokens mainly appear at the beginning of a code block. Inspired by the above findings, we propose a simple yet effective method: Adaptive Temperature (AdapT) sampling, which dynamically adjusts the temperature coefficient when decoding different tokens. We apply a larger temperature when sampling for challenging tokens, allowing LLMs to explore diverse choices. We employ a smaller temperature for confident tokens avoiding the influence of tail randomness noises. We apply AdapT sampling to LLMs with different sizes and conduct evaluations on two popular datasets. Results show that AdapT sampling significantly outperforms state-of-the-art decoding strategy.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 13 of its 21 references that are in the map. First posted on arXiv 6 Sep 2023.

Builds on

13 of 21 references.

References · 13

  1. 2021Program Synthesis with Large Language Modelscited by 3,509
  2. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  3. 2021Measuring Coding Challenge Competence With APPScited by 879
  4. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  5. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  6. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  7. 2017Beam Search Strategies for Neural Machine Translationcited by 232
  8. 2023Structured Chain-of-Thought Prompting for Code Generationcited by 136
  9. 2019Do Massively Pretrained Language Models Make Better Storytellers?cited by 90
  10. 2023SkCoder: A Sketch-based Approach for Automatic Code Generationcited by 40
  11. 2023AceCoder: Utilizing Existing Code to Enhance Code Generationcited by 25
  12. 2015On End-to-End Program Generation from User Intention by Deep Neural Networkscited by 6
1 more
  1. 2021Improving Diversity of Neural Text Generation via Inverse Probability Weightingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 19; 2026: 11.

37 papers.

Sort:

By area:

  1. 2025Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scalingcited by 93
  2. 2024Robustifying Safety-Aligned Large Language Models through Clean Data Curationcited by 43
  3. 2025Exploring the Impact of Temperature on Large Language Models:Hot or Cold?cited by 40
  4. 2026Look Inward to Explore Outward: Learning Temperature Policy from LLM Internal States via Hierarchical RLcited by 27
  5. 2025What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyondcited by 22
  6. 2024Proof Automation with Large Language Modelscited by 20
  7. 2024Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Modelscited by 15
  8. 2025Beyond Exponential Decay: Rethinking Error Accumulation in Large Language Modelscited by 11
  9. 2025Klear-CodeTest: Scalable Test Case Generation for Code Reinforcement Learningcited by 9
  10. 2025MS-BART: Unified Modeling of Mass Spectra and Molecules for Structure Elucidationcited by 9
  11. 2025The Consistency Hypothesis in Uncertainty Quantification for Large Language Modelscited by 6
  12. 2025CyLens: Towards Reinventing Cyber Threat Intelligence in the Paradigm of Agentic Large Language Modelscited by 5
25 more
  1. 2025Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMscited by 5
  2. 2025TreeCoder: Systematic Exploration and Optimisation of Decoding and Constraints for LLM Code Generationcited by 5
  3. 2025On LLMs' Internal Representation of Code Correctnesscited by 5
  4. 2024What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworkscited by 4
  5. 2024Entropy Controllable Direct Preference Optimizationcited by 4
  6. 2025From Token to Line: Enhancing Code Generation with a Long-Term Perspectivecited by 4
  7. 2025DecoRTL: A Run-time Decoding Framework for RTL Code Generation with LLMscited by 4
  8. 2024LLMs Prompted for Graphs: Hallucinations and Generative Capabilitiescited by 3
  9. 2024Improving Medical Diagnostics with Vision-Language Models: Convex Hull-Based Uncertainty Analysiscited by 3
  10. 2025From Recall to Reasoning: Automated Question Generation for Deeper Math Learning through Large Language Modelscited by 3
  11. 2025Automated Extraction of Material Properties using LLM-based AI Agentscited by 3
  12. 2026ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generationcited by 3
  13. 2026Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Modelscited by 3
  14. 2026Restoring Exploration after Post-Training: Latent Exploration Decoding for Large Reasoning Modelscited by 3
  15. 2026SCALE: Self-uncertainty Conditioned Adaptive Looking and Execution for Vision-Language-Action Modelscited by 3
  16. 2025Context-Enhanced Contrastive Search for Improved LLM Text Generationcited by 2
  17. 2025Confidence-Modulated Speculative Decoding for Large Language Modelscited by 1
  18. 2025Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoningcited by 1
  19. 2026The Ultimate Tutorial for AI-driven Scale Development in Generative Psychometrics: Releasing AIGENIE from its Bottlecited by 1
  20. 2026Configuration Over Selection: Hyperparameter Sensitivity Exceeds Model Differences in Open-Source LLMs for RTL Generationcited by 1
  21. 2026Generative Criticality in Large Language Model Temperature Scalingcited by 1
  22. 2025On Next-Token Prediction in LLMs: How End Goals Determine the Consistency of Decoding Algorithmscited by 0
  23. 2026Temperature Scaling Attack Disrupting Model Confidence in Federated Learningcited by 0
  24. 2026Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimizationcited by 0
  25. 2026FLEET: From Logits Entropy to Enhanced Trajectories in Text Generationcited by 0