# Exploring the Impact of Temperature on Large Language Models:Hot or Cold?

arXiv: https://arxiv.org/abs/2506.07295 · HTML: https://mlsanity.com/paper/2506.07295/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 8 Jun 2025.

## Builds on (most cited first, 20 of 20)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://mlsanity.com/paper/2306.00978/) (2023)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2304.06364/) (2023)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [The Effect of Sampling Temperature on Problem Solving in Large Language Models](https://mlsanity.com/paper/2402.05201/) (2024)
- [InFoBench: Evaluating Instruction Following Ability in Large Language Models](https://mlsanity.com/paper/2401.03601/) (2024)
- [Language GANs Falling Short](https://mlsanity.com/paper/1811.02549/) (2018)
- [Is Temperature the Creativity Parameter of Large Language Models?](https://mlsanity.com/paper/2405.00492/) (2024)
- [Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models](https://mlsanity.com/paper/2309.02772/) (2023)
- [This is not correct! Negation-aware Evaluation of Language Generation Systems](https://mlsanity.com/paper/2307.13989/) (2023)
- [KL-Divergence Guided Temperature Sampling](https://mlsanity.com/paper/2306.01286/) (2023)

## Built on by (most cited first, 30 of 40)

- [The threat of analytic flexibility in using large language models to simulate human data](https://mlsanity.com/paper/2509.13397/) (2025)
- [Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking](https://mlsanity.com/paper/2604.11581/) (2026)
- [What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversity](https://mlsanity.com/paper/2511.15593/) (2025)
- [On Calibration of Large Language Models: From Response To Capability](https://mlsanity.com/paper/2602.13540/) (2026)
- [Natural Language Instructions for Scene-Responsive Human-in-the-Loop Motion Planning in Autonomous Driving using Vision-Language-Action Models](https://mlsanity.com/paper/2602.04184/) (2026)
- [From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoning](https://mlsanity.com/paper/2606.18089/) (2026)
- [Agent Skill Framework: Perspectives on the Potential of Small to Medium Language Models in Industrial Environments](https://mlsanity.com/paper/2602.16653/) (2026)
- [The Necessity of Setting Temperature in LLM-as-a-Judge](https://mlsanity.com/paper/2603.28304/) (2026)
- [Whose Name Comes Up? II: Benchmarking and Intervention-Based Auditing of LLM-Based Scholar Recommendation](https://mlsanity.com/paper/2602.08873/) (2026)
- [Introducing Background Temperature to Characterise Hidden Randomness in Large Language Models](https://mlsanity.com/paper/2604.22411/) (2026)
- [Fine-grained Claim-level RAG Benchmark for Law](https://mlsanity.com/paper/2605.21071/) (2026)
- [CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer](https://mlsanity.com/paper/2510.00501/) (2025)
- [A Cross-Lingual Analysis of Bias in Large Language Models Using Romanian History](https://mlsanity.com/paper/2510.02362/) (2025)
- [Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization](https://mlsanity.com/paper/2510.05342/) (2025)
- [Do MLLMs See What We See? Analyzing Visualization Literacy Barriers in AI Systems](https://mlsanity.com/paper/2601.12585/) (2026)
- [Can LLM Aid in Solving Constraints with Inductive Definitions?](https://mlsanity.com/paper/2603.03668/) (2026)
- [An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMs](https://mlsanity.com/paper/2603.05400/) (2026)
- [The production of meaning in the processing of natural language](https://mlsanity.com/paper/2603.20381/) (2026)
- [When Hate Meets Facts: LLMs-in-the-Loop for Check-worthiness Detection in Hate Speech](https://mlsanity.com/paper/2603.25269/) (2026)
- [Generative AI for Visualizing Highway Construction Hazards Through Synthetic Images and Temporal Sequences](https://mlsanity.com/paper/2605.11276/) (2026)
- [Persona Conditioning of Brand Recommendations in Retrieval-Augmented Commercial Chat: A Prominence-Stratified Cross-Provider Audit](https://mlsanity.com/paper/2605.30207/) (2026)
- [Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026](https://mlsanity.com/paper/2606.04730/) (2026)
- [Large Language Models Acing Chartered Accountancy](https://mlsanity.com/paper/2506.21031/) (2025)
- [Architecting software monitors for control-flow anomaly detection through large language models and conformance checking](https://mlsanity.com/paper/2511.10876/) (2025)
- [NegBLEURT Forest: Leveraging Inconsistencies for Detecting Jailbreak Attacks](https://mlsanity.com/paper/2511.11784/) (2025)
- [Temperature in SLMs: Impact on Incident Categorization in On-Premises Environments](https://mlsanity.com/paper/2511.19464/) (2025)
- [TempPerturb-Eval: On the Joint Effects of Internal Temperature and External Perturbations in RAG Robustness](https://mlsanity.com/paper/2512.01183/) (2025)
- [Measuring Stability Beyond Accuracy in Small Open-Source Medical Large Language Models for Pediatric Endocrinology](https://mlsanity.com/paper/2601.11567/) (2025)
- [Automating the Detection of Requirement Dependencies Using Large Language Models](https://mlsanity.com/paper/2602.22456/) (2026)
- [Toward Epistemic Stability: Engineering Consistent Procedures for Industrial LLM Hallucination Reduction](https://mlsanity.com/paper/2603.10047/) (2026)

## Abstract (arXiv)

The sampling temperature, a critical hyperparameter in large language models (LLMs), modifies the logits before the softmax layer, thereby reshaping the distribution of output tokens. Recent studies have challenged the Stochastic Parrots analogy by demonstrating that LLMs are capable of understanding semantics rather than merely memorizing data and that randomness, modulated by sampling temperature, plays a crucial role in model inference. In this study, we systematically evaluated the impact of temperature in the range of 0 to 2 on data sets designed to assess six different capabilities, conducting statistical analyses on open source models of three different sizes: small (1B--4B), medium (6B--13B), and large (40B--80B). Our findings reveal distinct skill-specific effects of temperature on model performance, highlighting the complexity of optimal temperature selection in practical applications. To address this challenge, we propose a BERT-based temperature selector that takes advantage of these observed effects to identify the optimal temperature for a given prompt. We demonstrate that this approach can significantly improve the performance of small and medium models in the SuperGLUE datasets. Furthermore, our study extends to FP16 precision inference, revealing that temperature effects are consistent with those observed in 4-bit quantized models. By evaluating temperature effects up to 4.0 in three quantized models, we find that the Mutation Temperature -- the point at which significant performance changes occur -- increases with model size.