Eliminating Hallucination-Induced Errors in LLM Code Generation with Functional Clustering
Not on the map. Why?
Abstract (arXiv)
Modern code-generation LLMs can already solve a large fraction of programming problems, yet they still hallucinate subtle bugs that make their outputs unsafe for autonomous deployment. We present functional clustering, a black-box wrapper that eliminates nearly all hallucination-induced errors while providing a tunable confidence score. The wrapper samples many candidate programs, executes each on a self-generated test suite, and clusters candidates whose I/O behavior is identical; the empirical mass of the largest cluster serves as an exact confidence estimate. A single scalar threshold on this estimate lets users trade coverage for reliability with exponential guarantees. On LiveCodeBench our verifier preserves baseline pass@1 on solvable tasks yet slashes the error rate of returned answers from ~65% to 2%, and drives it to 0% at a conservative threshold while still answering 15.6% of prompts. Manual audits show that the few residual mistakes stem from prompt misinterpretation, not random generation noise, narrowing future work to specification clarity. Because the method requires only sampling and sandbox execution, it applies unchanged to closed-source APIs and future models, offering a practical path toward dependable, autonomous code generation. Our code is available on Github (https://github.com/20ChaituR/functional-clustering).
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 16 May 2025.
Builds on
22 of 34 references.
References · 22
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2024GPT-4o System Cardcited by 4,467
- 2023A Survey of Large Language Modelscited by 2,537
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
- 2022Competition-Level Code Generation with AlphaCodecited by 1,427
- 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
- 2021Measuring Coding Challenge Competence With APPScited by 879
- 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
- 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
- 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
10 more
- 2020Calibration of Pre-trained Transformerscited by 307
- 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
- 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolutioncited by 132
- 2024Benchmarking LLMs via Uncertainty Quantificationcited by 124
- 2025Competitive Programming with Large Reasoning Modelscited by 115
- 2024Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Codecited by 106
- 2024Mitigating LLM Hallucinations via Conformal Abstentioncited by 56
- 2023LitCab: Lightweight Language Model Calibration over Short- and Long-form Responsescited by 44
- 2024CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verificationcited by 32
- 2025Assessing Correctness in LLM-Based Code Generation via Uncertainty Estimationcited by 14
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.