# We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs

arXiv: https://arxiv.org/abs/2406.10279 · HTML: https://mlsanity.com/paper/2406.10279/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 97 references that are in the map.
- First posted on arXiv 12 Jun 2024.

## Builds on (most cited first, 30 of 53)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study](https://mlsanity.com/paper/2305.13860/) (2023)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Multi-step Jailbreaking Privacy Attacks on ChatGPT](https://mlsanity.com/paper/2304.05197/) (2023)
- [AudioGen: Textually Guided Audio Generation](https://mlsanity.com/paper/2209.15352/) (2022)
- [Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks](https://mlsanity.com/paper/2302.05733/) (2023)
- [A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models](https://mlsanity.com/paper/2401.01313/) (2024)

## Built on by (most cited first, 30 of 65)

- [Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey](https://mlsanity.com/paper/2412.20367/) (2024)
- [Design Patterns for Securing LLM Agents against Prompt Injections](https://mlsanity.com/paper/2506.08837/) (2025)
- [Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems](https://mlsanity.com/paper/2601.17548/) (2026)
- [The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey](https://mlsanity.com/paper/2603.11088/) (2026)
- [AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents](https://mlsanity.com/paper/2509.07764/) (2025)
- [DarkMind: Latent Chain-of-Thought Backdoor in Customized LLMs](https://mlsanity.com/paper/2501.18617/) (2025)
- [SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills](https://mlsanity.com/paper/2604.06550/) (2026)
- [Hallucination by Code Generation LLMs: Taxonomy, Benchmarks, Mitigation, and Challenges](https://mlsanity.com/paper/2504.20799/) (2025)
- [Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation](https://mlsanity.com/paper/2505.23657/) (2025)
- [SOK: A Taxonomy of Attack Vectors and Defense Strategies for Agentic Supply Chain Runtime](https://mlsanity.com/paper/2602.19555/) (2026)
- [A Unified Definition of Hallucination: It's The World Model, Stupid!](https://mlsanity.com/paper/2512.21577/) (2025)
- [Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents](https://mlsanity.com/paper/2606.05391/) (2026)
- [ETF: An Entity Tracing Framework for Hallucination Detection in Code Summaries](https://mlsanity.com/paper/2410.14748/) (2024)
- [Trust Me, Import This: Dependency Steering Attacks via Malicious Agent Skills](https://mlsanity.com/paper/2605.09594/) (2026)
- [RAG-Pull: Turning Retrieval into a Code-Injection Channel via Invisible Unicode Perturbations](https://mlsanity.com/paper/2510.11195/) (2025)
- [What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants](https://mlsanity.com/paper/2605.30777/) (2026)
- [Hierarchical Memorization in Large Language Models: Evidence from Citation Generation](https://mlsanity.com/paper/2511.08877/) (2025)
- [CoTDeceptor:Adversarial Code Obfuscation Against CoT-Enhanced LLM Code Agents](https://mlsanity.com/paper/2512.21250/) (2025)
- [Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Review](https://mlsanity.com/paper/2603.18740/) (2026)
- [A Comprehensive Framework for Evaluating API-oriented Code Generation in Large Language Models](https://mlsanity.com/paper/2409.15228/) (2024)
- [Empirical Evaluation of AI-Assisted Software Package Selection: A Knowledge Graph Approach](https://mlsanity.com/paper/2508.05693/) (2025)
- [Hallucinations in Bibliographic Recommendation: Citation Frequency as a Proxy for Training Data Redundancy](https://mlsanity.com/paper/2510.25378/) (2025)
- [Hallucinating AI Hijacking Attack: Large Language Models and Malicious Code Recommenders](https://mlsanity.com/paper/2410.06462/) (2024)
- [VECSR: Virtually Embodied Common Sense Reasoning System](https://mlsanity.com/paper/2505.02144/) (2025)
- [Investigating Security Implications of Automatically Generated Code on the Software Supply Chain](https://mlsanity.com/paper/2509.20277/) (2025)
- [Library Hallucinations in LLM-Generated Code: A Risk Analysis Grounded in Developer Queries](https://mlsanity.com/paper/2509.22202/) (2025)
- [HFuzzer: Testing Large Language Models for Package Hallucinations via Phrase-based Fuzzing](https://mlsanity.com/paper/2509.23835/) (2025)
- [Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization](https://mlsanity.com/paper/2510.05342/) (2025)
- [Hound: Relation-First Knowledge Graphs for Complex-System Reasoning in Security Audits](https://mlsanity.com/paper/2510.09633/) (2025)
- [LLMs as Packagers of HPC Software](https://mlsanity.com/paper/2511.05626/) (2025)

## Abstract (arXiv)

The reliance of popular programming languages such as Python and JavaScript on centralized package repositories and open-source software, combined with the emergence of code-generating Large Language Models (LLMs), has created a new type of threat to the software supply chain: package hallucinations. These hallucinations, which arise from fact-conflicting errors when generating code using LLMs, represent a novel form of package confusion attack that poses a critical threat to the integrity of the software supply chain. This paper conducts a rigorous and comprehensive evaluation of package hallucinations across different programming languages, settings, and parameters, exploring how a diverse set of models and configurations affect the likelihood of generating erroneous package recommendations and identifying the root causes of this phenomenon. Using 16 popular LLMs for code generation and two unique prompt datasets, we generate 576,000 code samples in two programming languages that we analyze for package hallucinations. Our findings reveal that that the average percentage of hallucinated packages is at least 5.2% for commercial models and 21.7% for open-source models, including a staggering 205,474 unique examples of hallucinated package names, further underscoring the severity and pervasiveness of this threat. To overcome this problem, we implement several hallucination mitigation strategies and show that they are able to significantly reduce the number of package hallucinations while maintaining code quality. Our experiments and findings highlight package hallucinations as a persistent and systemic phenomenon while using state-of-the-art LLMs for code generation, and a significant challenge which deserves the research community's urgent attention.