# Characterizing Overconfident Failure in LLM-Based Code Generation

arXiv: https://arxiv.org/abs/2610.11300 · HTML: https://mlsanity.com/paper/2610.11300/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 73 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Builds on (most cited first, 29 of 29)

- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models](https://mlsanity.com/paper/2305.19187/) (2023)
- [OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement](https://mlsanity.com/paper/2402.14658/) (2024)
- [Why Language Models Hallucinate](https://mlsanity.com/paper/2509.04664/) (2025)
- [Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification](https://mlsanity.com/paper/2403.04696/) (2024)
- [Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models](https://mlsanity.com/paper/2307.01379/) (2023)
- [Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Code](https://mlsanity.com/paper/2404.00971/) (2024)
- [Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph](https://mlsanity.com/paper/2406.15627/) (2024)
- [Mutual Information Alleviates Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2210.13210/) (2022)
- [A Survey of Uncertainty Estimation Methods on Large Language Models](https://mlsanity.com/paper/2503.00172/) (2025)
- [Uncertainty in Natural Language Generation: From Theory to Applications](https://mlsanity.com/paper/2307.15703/) (2023)
- [Calibration and Correctness of Language Models for Code](https://mlsanity.com/paper/2402.02047/) (2024)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [TESTEVAL: Benchmarking Large Language Models for Test Case Generation](https://mlsanity.com/paper/2406.04531/) (2024)
- [Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data](https://mlsanity.com/paper/2212.09171/) (2022)
- [From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered](https://mlsanity.com/paper/2506.07461/) (2025)
- [An Empirical Analysis of Uncertainty in Large Language Model Evaluations](https://mlsanity.com/paper/2502.10709/) (2025)
- [Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in Its Latent Thoughts](https://mlsanity.com/paper/2509.26314/) (2025)
- [Reconsidering LLM Uncertainty Estimation Methods in the Wild](https://mlsanity.com/paper/2506.01114/) (2025)
- [Assessing Correctness in LLM-Based Code Generation via Uncertainty Estimation](https://mlsanity.com/paper/2502.11620/) (2025)
- [Training Language Models to Generate Quality Code with Program Analysis Feedback](https://mlsanity.com/paper/2505.22704/) (2025)
- [Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style](https://mlsanity.com/paper/2409.10955/) (2024)
- [R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents](https://mlsanity.com/paper/2303.00732/) (2023)
- [AmbigNLG: Addressing Task Ambiguity in Instruction for NLG](https://mlsanity.com/paper/2402.17717/) (2024)
- [Estimating Predictive Uncertainty Under Program Data Distribution Shift](https://mlsanity.com/paper/2107.10989/) (2021)
- [DOCE: Finding the Sweet Spot for Execution-Based Code Generation](https://mlsanity.com/paper/2408.13745/) (2024)
- [Code Is More Than Text: Uncertainty Estimation for Code Generation](https://mlsanity.com/paper/2606.09577/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used for automated code generation, but generated programs can appear syntactically plausible while still failing execution-based correctness checks. Existing validation methods, such as testing and program analysis, remain essential but are often incomplete, costly, or applied only after generation. Model-derived uncertainty is therefore a natural early reliability signal. This paper studies the dilemma of overconfidence in code LLMs where incorrect programs are often generated with token-level confidence comparable to correct programs. We study this dilemma across four open-source code models and three execution-based benchmarks. Our analysis begins by investigating whether existing uncertainty metrics provide reliable proxies for execution correctness in code generation. We then characterize overconfidence at both global and local token levels, asking whether incorrect programs remain indistinguishable from correct ones under confidence and entropy summaries, including selective generation and the limits of instruction tuning. Finally, we evaluate whether common mitigation strategies reduce this failure mode. Our study yields four findings. First, existing uncertainty signals provide only partial and model-dependent evidence of execution failure. Second, overconfidence persists at both program and token levels, and uncertainty-based selection does not consistently improve accepted-set accuracy. Third, instruction tuning can increase certainty on failing generations without consistently improving correctness discrimination. Fourth, common mitigation techniques improve specific aspects of reliability but do not reliably resolve overconfident failure. Our exploratory latent analysis suggests that hidden representations may encode correctness-related signals that output confidence does not expose.