# Investigating Security Implications of Automatically Generated Code on the Software Supply Chain

arXiv: https://arxiv.org/abs/2509.20277 · HTML: https://mlsanity.com/paper/2509.20277/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 73 references that are in the map.
- First posted on arXiv 24 Sep 2025.

## Builds on (most cited first, 28 of 28)

- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- ["Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models](https://mlsanity.com/paper/2308.03825/) (2023)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023)
- [Chain-of-Verification Reduces Hallucination in Large Language Models](https://mlsanity.com/paper/2309.11495/) (2023)
- [You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion](https://mlsanity.com/paper/2007.02220/) (2020)
- [Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Code](https://mlsanity.com/paper/2404.00971/) (2024)
- [LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation](https://mlsanity.com/paper/2409.20550/) (2024)
- [Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants](https://mlsanity.com/paper/2208.09727/) (2022)
- [Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues](https://mlsanity.com/paper/2307.12596/) (2023)
- [Can ChatGPT replace StackOverflow? A Study on Robustness and Reliability of Large Language Model Code Generation](https://mlsanity.com/paper/2308.10335/) (2023)
- [Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT](https://mlsanity.com/paper/2304.10778/) (2023)
- [We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs](https://mlsanity.com/paper/2406.10279/) (2024)
- [TrojanPuzzle: Covertly Poisoning Code-Suggestion Models](https://mlsanity.com/paper/2301.02344/) (2023)
- [Bugs in Large Language Models Generated Code: An Empirical Study](https://mlsanity.com/paper/2403.08937/) (2024)
- [Taxonomy of Attacks on Open-Source Software Supply Chains](https://mlsanity.com/paper/2204.04008/) (2022)
- [No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT](https://mlsanity.com/paper/2308.04838/) (2023)
- [Large Language Models and Simple, Stupid Bugs](https://mlsanity.com/paper/2303.11455/) (2023)
- [An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection](https://mlsanity.com/paper/2406.06822/) (2024)
- [An Empirical Comparison of Dependency Network Evolution in Seven Software Packaging Ecosystems](https://mlsanity.com/paper/1710.04936/) (2017)
- [What are Weak Links in the npm Supply Chain?](https://mlsanity.com/paper/2112.10165/) (2021)
- [Stack Overflow Considered Harmful? The Impact of Copy&Paste on Android Application Security](https://mlsanity.com/paper/1710.03135/) (2017)
- [On the effectiveness of Large Language Models for GitHub Workflows](https://mlsanity.com/paper/2403.12446/) (2024)
- [Do Software Security Practices Yield Fewer Vulnerabilities?](https://mlsanity.com/paper/2210.14884/) (2022)
- [Thou Shalt Not Depend on Me: Analysing the Use of Outdated JavaScript Libraries on the Web](https://mlsanity.com/paper/1811.00918/) (2018)
- [What's in a Name? Understanding Profile Name Reuse on Twitter](https://mlsanity.com/paper/1702.04256/) (2017)

## Built on by (most cited first, 1 of 1)

- [Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks via Universal and Transferable Adversarial HalluSquatting](https://mlsanity.com/paper/2607.07433/) (2026)

## Abstract (arXiv)

In recent years, various software supply chain (SSC) attacks have posed significant risks to the global community. Severe consequences may arise if developers integrate insecure code snippets that are vulnerable to SSC attacks into their products. Particularly, code generation techniques, such as large language models (LLMs), have been widely utilized in the developer community. However, LLMs are known to suffer from inherent issues when generating code, including fabrication, misinformation, and reliance on outdated training data, all of which can result in serious software supply chain threats. In this paper, we investigate the security threats to the SSC that arise from these inherent issues. We examine three categories of threats, including eleven potential SSC-related threats, related to external components in source code, and continuous integration configuration files. We find some threats in LLM-generated code could enable attackers to hijack software and workflows, while some others might cause potential hidden threats that compromise the security of the software over time. To understand these security impacts and severity, we design a tool, SSCGuard, to generate 439,138 prompts based on SSC-related questions collected online, and analyze the responses of four popular LLMs from GPT and Llama. Our results show that all identified SSC-related threats persistently exist. To mitigate these risks, we propose a novel prompt-based defense mechanism, namely Chain-of-Confirmation, to reduce fabrication, and a middleware-based defense that informs users of various SSC threats.