# An Empirical Analysis of Static Analysis Methods for Detection and Mitigation of Code Library Hallucinations

arXiv: https://arxiv.org/abs/2604.07755 · HTML: https://mlsanity.com/paper/2604.07755/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 33 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 27 of 27)

- [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022)
- [Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning](https://mlsanity.com/paper/2305.13971/) (2023)
- [Is Self-Repair a Silver Bullet for Code Generation?](https://mlsanity.com/paper/2306.09896/) (2023)
- [Granite Code Models: A Family of Open Foundation Models for Code Intelligence](https://mlsanity.com/paper/2405.04324/) (2024)
- [Self-Edit: Fault-Aware Code Editor for Code Generation](https://mlsanity.com/paper/2305.04087/) (2023)
- [Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation](https://mlsanity.com/paper/2403.06988/) (2024)
- [Reducing hallucination in structured outputs via Retrieval-Augmented Generation](https://mlsanity.com/paper/2404.08189/) (2024)
- [Execution-Based Evaluation for Open-Domain Code Generation](https://mlsanity.com/paper/2212.10481/) (2022)
- [CodeRAG-Bench: Can Retrieval Augment Code Generation?](https://mlsanity.com/paper/2406.14497/) (2024)
- [Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues](https://mlsanity.com/paper/2307.12596/) (2023)
- [SynCode: LLM Generation with Grammar Augmentation](https://mlsanity.com/paper/2403.01632/) (2024)
- [We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs](https://mlsanity.com/paper/2406.10279/) (2024)
- [NExT: Teaching Large Language Models to Reason about Code Execution](https://mlsanity.com/paper/2404.14662/) (2024)
- [Automata-based constraints for language model decoding](https://mlsanity.com/paper/2407.08103/) (2024)
- [Constrained Decoding for Secure Code Generation](https://mlsanity.com/paper/2405.00218/) (2024)
- [HALoGEN: Fantastic LLM Hallucinations and Where to Find Them](https://mlsanity.com/paper/2501.08292/) (2025)
- [FixEval: Execution-based Evaluation of Program Fixes for Programming Problems](https://mlsanity.com/paper/2206.07796/) (2022)
- [De-Hallucinator: Mitigating LLM Hallucinations in Code Generation Tasks via Iterative Grounding](https://mlsanity.com/paper/2401.01701/) (2024)
- [Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities](https://mlsanity.com/paper/2501.19012/) (2025)
- [A Static Evaluation of Code Completion by Large Language Models](https://mlsanity.com/paper/2306.03203/) (2023)
- [Collu-Bench: A Benchmark for Predicting Language Model Hallucinations in Code](https://mlsanity.com/paper/2410.09997/) (2024)
- [FLAP: Flow-Adhering Planning with Constrained Decoding in LLMs](https://mlsanity.com/paper/2403.05766/) (2024)
- [Constrained Decoding for Fill-in-the-Middle Code Language Models via Efficient Left and Right Quotienting of Context-Sensitive Grammars](https://mlsanity.com/paper/2402.17988/) (2024)
- [Towards Mitigating API Hallucination in Code Generated by LLMs with Hierarchical Dependency Aware](https://mlsanity.com/paper/2505.05057/) (2025)
- [ChatGPT Incorrectness Detection in Software Reviews](https://mlsanity.com/paper/2403.16347/) (2024)
- [LLM-based Frameworks for API Argument Filling in Task-Oriented Conversational Systems](https://mlsanity.com/paper/2407.12016/) (2024)
- [The Power of Types: Exploring the Impact of Type Checking on Neural Bug Detection in Dynamically Typed Languages](https://mlsanity.com/paper/2411.15368/) (2024)

## Abstract (arXiv)

Despite extensive research, Large Language Models continue to hallucinate when generating code, particularly when using libraries. On NL-to-code benchmarks that require library use, we find that LLMs generate code that uses non-existent library features in 8.1-40% of responses. One intuitive approach for detection and mitigation of hallucinations is static analysis. In this paper, we analyse the potential of static analysis tools, both in terms of what they can solve and what they cannot. We find that static analysis tools can detect 16-70% of all errors, and 14-85% of library hallucinations, with performance varying by LLM and dataset. Through manual analysis, we identify cases a static method could not plausibly catch, which gives an upper bound on their potential from 48.5% to 77%. Overall, we show that static analysis methods are cheap method for addressing some forms of hallucination, and we quantify how far short of solving the problem they will always be.