mlsanityData to 7 Oct 2026

Calibration and Correctness of Language Models for Code

arXiv 2402.02047 ↗First posted 3 Feb 2024cs.SE cs.LGcited by 49

Not on the map. Why?

Abstract (arXiv)

Machine learning models are widely used, but can also often be wrong. Users would benefit from a reliable indication of whether a given output from a given model should be trusted, so a rational decision can be made whether to use the output or not. For example, outputs can be associated with a confidence measure; if this confidence measure is strongly associated with likelihood of correctness, then the model is said to be well-calibrated. A well-calibrated confidence measure can serve as a basis for rational, graduated decision-making on how much review and care is needed when using generated code. Calibration has so far been studied in mostly non-generative (e.g. classification) settings, especially in software engineering. However, generated code can quite often be wrong: Given generated code, developers must decide whether to use directly, use after varying intensity of careful review, or discard model-generated code. Thus, calibration is vital in generative settings. We make several contributions. We develop a framework for evaluating the calibration of code-generating models. We consider several tasks, correctness criteria, datasets, and approaches, and find that, by and large, generative code models we test are not well-calibrated out of the box. We then show how calibration can be improved using standard methods, such as Platt scaling. Since Platt scaling relies on the prior availability of correctness data, we evaluate the applicability and generalizability of Platt scaling in software engineering, discuss settings where it has good potential for practical use, and settings where it does not. Our contributions will lead to better-calibrated decision-making in the current use of code generated by language models, and offers a framework for future research to further improve calibration methods for generative models in software engineering.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 50 of its 83 references that are in the map. First posted on arXiv 3 Feb 2024.

Builds on

50 of 83 references.

References · 50

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2017On Calibration of Modern Neural Networkscited by 4,860
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  10. 2020Extracting Training Data from Large Language Modelscited by 2,144
  11. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  12. 2022Language Models (Mostly) Know What They Knowcited by 1,783
38 more
  1. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  2. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  3. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  4. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  5. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  6. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  7. 2019Measuring Calibration in Deep Learningcited by 443
  8. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  9. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  10. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  11. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
  12. 2020Calibration of Pre-trained Transformerscited by 307
  13. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  14. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
  15. 2023CodeGen2: Lessons for Training LLMs on Programming and Natural Languagescited by 124
  16. 2020You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completioncited by 112
  17. 2022Productivity Assessment of Neural Code Completioncited by 97
  18. 2023Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Modelscited by 91
  19. 2023Impact of Code Language Models on Automated Program Repaircited by 85
  20. 2022Coder Reviewer Reranking for Code Generationcited by 83
  21. 2022Automated Repair of Programs from Large Language Modelscited by 71
  22. 2020Code Prediction by Feeding Trees to Transformerscited by 70
  23. 2023A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trendscited by 66
  24. 2022Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?cited by 63
  25. 2022A Close Look into the Calibration of Pre-trained Language Modelscited by 57
  26. 2023A Survey of Learning-based Automated Program Repaircited by 55
  27. 2022CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequencescited by 49
  28. 2023Large Language Models and Simple, Stupid Bugscited by 45
  29. 2023LitCab: Lightweight Language Model Calibration over Short- and Long-form Responsescited by 44
  30. 2019How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Datasetcited by 35
  31. 2022On the Calibration of Pre-trained Language Models using Mixup Guided by Area Under the Margin and Saliencycited by 31
  32. 2023Better patching using LLM prompting, via Self-Consistencycited by 29
  33. 2023Trustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmapscited by 19
  34. 2022Toward Trustworthy Neural Program Synthesiscited by 11
  35. 2023R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intentscited by 10
  36. 2019Operational Calibration: Debugging Confidence Errors for DNNs in the Fieldcited by 6
  37. 2023AI Safety Subproblems for Software Engineering Researcherscited by 3
  38. 2023PACE-LM: Prompting and Augmentation for Calibrated Confidence Estimation with GPT-4 in Cloud Incident Root Cause Analysiscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 24; 2026: 15.

49 papers.

Sort:

By area:

  1. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  2. 2025Maximizing Confidence Alone Improves Reasoningcited by 97
  3. 2024You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projectscited by 45
  4. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 36
  5. 2025Uncertainty-Guided Chain-of-Thought for Code Generation with LLMscited by 30
  6. 2025Correctness Assessment of Code Generated by Large Language Models Using Internal Representationscited by 17
  7. 2025From Informal to Formal -- Incorporating and Evaluating LLMs on Natural Language Requirements to Verifiable Formal Proofscited by 17
  8. 2024Calibration of Large Language Models on Code Summarizationcited by 12
  9. 2025Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inferencecited by 11
  10. 2024Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanationscited by 10
  11. 2025SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQLcited by 10
  12. 2024Beyond Words: On Large Language Models Actionability in Mission-Critical Risk Analysiscited by 9
37 more
  1. 2025Do Large Language Models Know What They Are Capable Of?cited by 9
  2. 2024A Survey of Calibration Process for Black-Box LLMscited by 8
  3. 2025Risk Assessment Framework for Code LLMs via Leveraging Internal Statescited by 8
  4. 2025GPT-4.1 Sets the Standard in Automated Experiment Design Using Novel Python Librariescited by 8
  5. 2024Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systemscited by 7
  6. 2025Conformal Information Pursuit for Interactively Guiding Large Language Modelscited by 7
  7. 2025LLM-ProS: Analyzing Large Language Models' Performance in Competitive Problem Solvingcited by 6
  8. 2025Human-AI Experience in Integrated Development Environments: A Systematic Literature Reviewcited by 5
  9. 2025Large Language Models Are Unreliable for Cyber Threat Intelligencecited by 5
  10. 2025On LLMs' Internal Representation of Code Correctnesscited by 5
  11. 2024Showing LLM-Generated Code Selectively Based on Confidence of LLMscited by 4
  12. 2025The Fools are Certain; the Wise are Doubtful: Exploring LLM Confidence in Code Completioncited by 3
  13. 2025Localized Calibrated Uncertainty in Code Language Modelscited by 3
  14. 2026Functional Entropy: Predicting Functional Correctness in LLM-Generated Code with Uncertainty Quantificationcited by 3
  15. 2026When are likely answers right? On Sequence Probability and Correctness in LLMscited by 3
  16. 2025HASHIRU: Hierarchical Agent System for Hybrid Intelligent Resource Utilizationcited by 2
  17. 2025Multicalibration for LLM-based Code Generationcited by 2
  18. 2025Towards Functional Correctness of Large Code Models with Selective Generationcited by 1
  19. 2025Are LLM Agents the New RPA? A Comparative Study with RPA Across Enterprise Workflowscited by 1
  20. 2025Open the Oyster: Empirical Evaluation and Improvement of Code Reasoning Confidence in LLMscited by 1
  21. 2026Model See, Model Do? Exposure-Aware Evaluation of Bug-vs-Fix Preference in Code LLMscited by 1
  22. 2026Verifier-Guided Code Translation via Meta-Step Decodingcited by 1
  23. 2026FLARE: Fine-Grained Diagnostic Feedback for LLM Code Refinementcited by 1
  24. 2026On the Robustness of LLMs' Internal Representation of Code Correctnesscited by 1
  25. 2024Automated Creation of Source Code Variants of a Cryptographic Hash Function Implementation Using Generative Pre-Trained Transformer Modelscited by 0
  26. 2025Mapping the Trust Terrain: LLMs in Software Engineering -- Insights and Perspectivescited by 0
  27. 2025Does In-IDE Calibration of Large Language Models work at Scale?cited by 0
  28. 2025TaskEval: Synthesised Evaluation for Foundation-Model Taskscited by 0
  29. 2026Coding in a Bubble? Evaluating LLMs in Resolving Context Adaptation Bugs During Code Adaptationcited by 0
  30. 2026When Models Know When They Do Not Know: Calibration, Cascading, and Cleaningcited by 0
  31. 2026Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revisioncited by 0
  32. 2026PYTHALAB-MERA: Validation-Grounded Memory, Retrieval, and Acceptance Control for Frozen-LLM Coding Agentscited by 0
  33. 2026An Empirical Study of Security Calibration in Large Language Models for Codecited by 0
  34. 2026Code-MUE: Measuring Code LLMs' Uncertainty through Execution-based Semantic Interaction Graphscited by 0
  35. 2026SciCodePile: A 128GB Corpus and Executable Benchmark for Challenging Scientific Code Generationcited by 0
  36. 2026When Do Supervised UQ Ensembles Improve LLM Hallucination Detection? A Robustness Studycited by 0
  37. 2026Characterizing Overconfident Failure in LLM-Based Code Generationcited by 0