# Calibration and Correctness of Language Models for Code

arXiv: https://arxiv.org/abs/2402.02047 · HTML: https://mlsanity.com/paper/2402.02047/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 83 references that are in the map.
- First posted on arXiv 3 Feb 2024.

## Builds on (most cited first, 30 of 50)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [Revisiting the Calibration of Modern Neural Networks](https://mlsanity.com/paper/2106.07998/) (2021)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Calibration of Pre-trained Transformers](https://mlsanity.com/paper/2003.07892/) (2020)
- [LEVER: Learning to Verify Language-to-Code Generation with Execution](https://mlsanity.com/paper/2302.08468/) (2023)
- [Grounded Copilot: How Programmers Interact with Code-Generating Models](https://mlsanity.com/paper/2206.15000/) (2022)
- [You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion](https://mlsanity.com/paper/2007.02220/) (2020)
- [CodeGen2: Lessons for Training LLMs on Programming and Natural Languages](https://mlsanity.com/paper/2305.02309/) (2023)
- [Productivity Assessment of Neural Code Completion](https://mlsanity.com/paper/2205.06537/) (2022)
- [Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models](https://mlsanity.com/paper/2302.13439/) (2023)

## Built on by (most cited first, 30 of 49)

- [A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions](https://mlsanity.com/paper/2412.05563/) (2024)
- [Maximizing Confidence Alone Improves Reasoning](https://mlsanity.com/paper/2505.22660/) (2025)
- [You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects](https://mlsanity.com/paper/2412.10133/) (2024)
- [Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices](https://mlsanity.com/paper/2403.12503/) (2024)
- [Uncertainty-Guided Chain-of-Thought for Code Generation with LLMs](https://mlsanity.com/paper/2503.15341/) (2025)
- [From Informal to Formal -- Incorporating and Evaluating LLMs on Natural Language Requirements to Verifiable Formal Proofs](https://mlsanity.com/paper/2501.16207/) (2025)
- [Correctness Assessment of Code Generated by Large Language Models Using Internal Representations](https://mlsanity.com/paper/2501.12934/) (2025)
- [Calibration of Large Language Models on Code Summarization](https://mlsanity.com/paper/2404.19318/) (2024)
- [SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL](https://mlsanity.com/paper/2506.04494/) (2025)
- [Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanations](https://mlsanity.com/paper/2407.08983/) (2024)
- [Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference](https://mlsanity.com/paper/2506.21408/) (2025)
- [Do Large Language Models Know What They Are Capable Of?](https://mlsanity.com/paper/2512.24661/) (2025)
- [Beyond Words: On Large Language Models Actionability in Mission-Critical Risk Analysis](https://mlsanity.com/paper/2406.10273/) (2024)
- [Risk Assessment Framework for Code LLMs via Leveraging Internal States](https://mlsanity.com/paper/2504.14640/) (2025)
- [A Survey of Calibration Process for Black-Box LLMs](https://mlsanity.com/paper/2412.12767/) (2024)
- [GPT-4.1 Sets the Standard in Automated Experiment Design Using Novel Python Libraries](https://mlsanity.com/paper/2508.00033/) (2025)
- [Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2411.19463/) (2024)
- [Conformal Information Pursuit for Interactively Guiding Large Language Models](https://mlsanity.com/paper/2507.03279/) (2025)
- [LLM-ProS: Analyzing Large Language Models' Performance in Competitive Problem Solving](https://mlsanity.com/paper/2502.04355/) (2025)
- [Human-AI Experience in Integrated Development Environments: A Systematic Literature Review](https://mlsanity.com/paper/2503.06195/) (2025)
- [Large Language Models Are Unreliable for Cyber Threat Intelligence](https://mlsanity.com/paper/2503.23175/) (2025)
- [On LLMs' Internal Representation of Code Correctness](https://mlsanity.com/paper/2512.07404/) (2025)
- [Showing LLM-Generated Code Selectively Based on Confidence of LLMs](https://mlsanity.com/paper/2410.03234/) (2024)
- [When are likely answers right? On Sequence Probability and Correctness in LLMs](https://mlsanity.com/paper/2606.27359/) (2026)
- [The Fools are Certain; the Wise are Doubtful: Exploring LLM Confidence in Code Completion](https://mlsanity.com/paper/2508.16131/) (2025)
- [Localized Calibrated Uncertainty in Code Language Models](https://mlsanity.com/paper/2512.24560/) (2025)
- [Functional Entropy: Predicting Functional Correctness in LLM-Generated Code with Uncertainty Quantification](https://mlsanity.com/paper/2605.28500/) (2026)
- [HASHIRU: Hierarchical Agent System for Hybrid Intelligent Resource Utilization](https://mlsanity.com/paper/2506.04255/) (2025)
- [Multicalibration for LLM-based Code Generation](https://mlsanity.com/paper/2512.08810/) (2025)
- [Verifier-Guided Code Translation via Meta-Step Decoding](https://mlsanity.com/paper/2605.17626/) (2026)

## Abstract (arXiv)

Machine learning models are widely used, but can also often be wrong. Users would benefit from a reliable indication of whether a given output from a given model should be trusted, so a rational decision can be made whether to use the output or not. For example, outputs can be associated with a confidence measure; if this confidence measure is strongly associated with likelihood of correctness, then the model is said to be well-calibrated. A well-calibrated confidence measure can serve as a basis for rational, graduated decision-making on how much review and care is needed when using generated code. Calibration has so far been studied in mostly non-generative (e.g. classification) settings, especially in software engineering. However, generated code can quite often be wrong: Given generated code, developers must decide whether to use directly, use after varying intensity of careful review, or discard model-generated code. Thus, calibration is vital in generative settings. We make several contributions. We develop a framework for evaluating the calibration of code-generating models. We consider several tasks, correctness criteria, datasets, and approaches, and find that, by and large, generative code models we test are not well-calibrated out of the box. We then show how calibration can be improved using standard methods, such as Platt scaling. Since Platt scaling relies on the prior availability of correctness data, we evaluate the applicability and generalizability of Platt scaling in software engineering, discuss settings where it has good potential for practical use, and settings where it does not. Our contributions will lead to better-calibrated decision-making in the current use of code generated by language models, and offers a framework for future research to further improve calibration methods for generative models in software engineering.