mlsanityData to 7 Oct 2026

Code Is More Than Text: Uncertainty Estimation for Code Generation

arXiv 2606.09577 ↗First posted 8 Jun 2026cs.CL cs.LG cs.SEcited by 4

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are increasingly deployed as code generators, where silently wrong programs pose real safety and reliability risks. Reliable uncertainty estimation (UE) is essential for selective prediction, human-in-the-loop review, and downstream agentic decisions. Yet most existing code UE methods are inherited from natural language (NL) generation and ignore properties that make code distinct. We argue that code differs from NL in three ways: a single wrong token can break an entire program (token fragility); algorithmic intent and concrete implementation can disagree independently (intent-code gap); and programs can be executed (executability). We instantiate these properties as three orthogonal uncertainty axes: lexical (Top-K token entropy), algorithmic (pseudo-code consistency), and functional (behavioral consistency). Across five code LLMs, our three-axis ensemble improves average AUROC from 0.696 for the strongest NL-derived baseline to 0.776 (+8.1 points). Notably, on Qwen3-14B, our single-pass Top-K token entropy matches the strongest multi-pass baseline while being over 3x cheaper; across models, it remains a competitive low-cost signal. These results suggest that code UE deserves code-specific design rather than direct NL ports.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 21 of its 28 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

21 of 28 references.

References · 21

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2023Teaching Large Language Models to Self-Debugcited by 963
  5. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  6. 2021Measuring Coding Challenge Competence With APPScited by 879
  7. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  8. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  9. 2022CodeT: Code Generation with Generated Testscited by 369
  10. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  11. 2023Self-Edit: Fault-Aware Code Editor for Code Generationcited by 110
  12. 2024LUQ: Long-text Uncertainty Quantification for LLMscited by 79
9 more
  1. 2025SWE-Exp: Experience-Driven Software Issue Resolutioncited by 64
  2. 2025SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolutioncited by 52
  3. 2024From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debuggingcited by 46
  4. 2025Pruning the Unsurprising: Efficient LLM Reasoning via First-Token Surprisalcited by 32
  5. 2024Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmerscited by 27
  6. 2024Atomic Calibration of LLMs in Long-Form Generationscited by 23
  7. 2025Entropy-Gated Branching for Efficient Test-Time Reasoningcited by 21
  8. 2025Enhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approachcited by 10
  9. 2024Perplexed: Understanding When Large Language Models are Confusedcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026EarlyEval: Cheaper Agent Evaluation via Early Outcome Predictioncited by 2
  2. 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Studycited by 0
  3. 2026Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agentscited by 0
  4. 2026Characterizing Overconfident Failure in LLM-Based Code Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Ensemble-Based Uncertainty Estimation for Code Correctness Estimation ↗
  2. 2026ExeCRE: Execution-Consistency Guided Reliability Estimation for Self-Correcting Code Generation
  3. 2025Trusted Uncertainty in Large Language Models: A Unified Framework for Confidence Calibration and Risk-Controlled Refusal
  4. 2025EVALOOOP: A Self-Consistency-Centered Framework for Assessing Large Language Model Robustness in Programming
  5. 2026Think Before You Code: Dual Reasoning for the NLSafety-Utility Trade-Off in LLM Code Generation
  6. 2025Pragmatic Reasoning improves LLM Code Generation
  7. 2024Rethinking Uncertainty Estimation in LLMs: A Principled Single-Sequence Measure
  8. 2024Rethinking the Uncertainty: A Critical Review and Analysis in the Era of Large Language Models