# Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Code

arXiv: https://arxiv.org/abs/2404.00971 · HTML: https://mlsanity.com/paper/2404.00971/

## Key facts

- Cited by 107 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 87 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 30 of 53)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022)
- [Fine-Grained Human Feedback Gives Better Rewards for Language Model Training](https://mlsanity.com/paper/2306.01693/) (2023)
- [RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation](https://mlsanity.com/paper/2303.12570/) (2023)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback](https://mlsanity.com/paper/2302.12813/) (2023)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [How Language Model Hallucinations Can Snowball](https://mlsanity.com/paper/2305.13534/) (2023)
- [Unsupervised Translation of Programming Languages](https://mlsanity.com/paper/2006.03511/) (2020)

## Built on by (most cited first, 30 of 107)

- [SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation](https://mlsanity.com/paper/2410.12761/) (2024)
- [LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation](https://mlsanity.com/paper/2409.20550/) (2024)
- [We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs](https://mlsanity.com/paper/2406.10279/) (2024)
- [Agents in Software Engineering: Survey, Landscape, and Vision](https://mlsanity.com/paper/2409.09030/) (2024)
- [Exploring and Unleashing the Power of Large Language Models in Automated Code Translation](https://mlsanity.com/paper/2404.14646/) (2024)
- [What's Wrong with Your Code Generated by Large Language Models? An Extensive Study](https://mlsanity.com/paper/2407.06153/) (2024)
- [Context-DPO: Aligning Language Models for Context-Faithfulness](https://mlsanity.com/paper/2412.15280/) (2024)
- [Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models](https://mlsanity.com/paper/2503.15888/) (2025)
- [A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learning](https://mlsanity.com/paper/2410.14660/) (2024)
- [A Survey of Useful LLM Evaluation](https://mlsanity.com/paper/2406.00936/) (2024)
- [AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities](https://mlsanity.com/paper/2508.11126/) (2025)
- [SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared Task on Hallucinations and Related Observable Overgeneration Mistakes](https://mlsanity.com/paper/2504.11975/) (2025)
- [Security of AI Agents](https://mlsanity.com/paper/2406.08689/) (2024)
- [Estimating LLM Uncertainty with Evidence](https://mlsanity.com/paper/2502.00290/) (2025)
- [ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generation](https://mlsanity.com/paper/2503.07010/) (2025)
- [Rome was Not Built in a Single Step: Hierarchical Prompting for LLM-based Chip Design](https://mlsanity.com/paper/2407.18276/) (2024)
- [On Mitigating Code LLM Hallucinations with API Documentation](https://mlsanity.com/paper/2407.09726/) (2024)
- [An LLM Agent for Automatic Geospatial Data Analysis](https://mlsanity.com/paper/2410.18792/) (2024)
- [VeriPlan: Integrating Formal Verification and LLMs into End-User Planning](https://mlsanity.com/paper/2502.17898/) (2025)
- [Cost-Effective Hallucination Detection for LLMs](https://mlsanity.com/paper/2407.21424/) (2024)
- [CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMs](https://mlsanity.com/paper/2410.01999/) (2024)
- [Automated test generation to evaluate tool-augmented LLMs as conversational AI agents](https://mlsanity.com/paper/2409.15934/) (2024)
- [Assessing and Verifying Task Utility in LLM-Powered Applications](https://mlsanity.com/paper/2405.02178/) (2024)
- [Towards Rationality in Language and Multimodal Agents: A Survey](https://mlsanity.com/paper/2406.00252/) (2024)
- [An Empirical Study on Failures in Automated Issue Solving](https://mlsanity.com/paper/2509.13941/) (2025)
- [Trust Region Masking for Long-Horizon LLM Reinforcement Learning](https://mlsanity.com/paper/2512.23075/) (2025)
- [Are LLMs Correctly Integrated into Software Systems?](https://mlsanity.com/paper/2407.05138/) (2024)
- [Collu-Bench: A Benchmark for Predicting Language Model Hallucinations in Code](https://mlsanity.com/paper/2410.09997/) (2024)
- [DeepSeek vs. ChatGPT vs. Claude: A Comparative Study for Scientific Computing and Scientific Machine Learning Tasks](https://mlsanity.com/paper/2502.17764/) (2025)
- [HiBench: Benchmarking LLMs Capability on Hierarchical Structure Reasoning](https://mlsanity.com/paper/2503.00912/) (2025)

## Abstract (arXiv)

The rise of Large Language Models (LLMs) has significantly advanced various applications on software engineering tasks, particularly in code generation. Despite the promising performance, LLMs are prone to generate hallucinations, which means LLMs might produce outputs that deviate from users' intent, exhibit internal inconsistencies, or misaligned with the real-world knowledge, making the deployment of LLMs potentially risky in a wide range of applications. Existing work mainly focuses on investigating the hallucination in the domain of Natural Language Generation (NLG), leaving a gap in comprehensively understanding the types, causes, and impacts of hallucinations in the context of code generation. To bridge the gap, we conducted a thematic analysis of the LLM-generated code to summarize and categorize the hallucinations, as well as their causes and impacts. Our study established a comprehensive taxonomy of code hallucinations, encompassing 3 primary categories and 12 specific categories. Furthermore, we systematically analyzed the distribution of hallucinations, exploring variations among different LLMs and benchmarks. Moreover, we perform an in-depth analysis on the causes and impacts of various hallucinations, aiming to provide valuable insights into hallucination mitigation. Finally, to enhance the correctness and reliability of LLM-generated code in a lightweight manner, we explore training-free hallucination mitigation approaches by prompt enhancing techniques. We believe our findings will shed light on future research about code hallucination evaluation and mitigation, ultimately paving the way for building more effective and reliable code LLMs in the future. The replication package is available at https://github.com/Lorien1128/code_hallucination