mlsanityData to 7 Oct 2026

Security of LLM-generated Code: A Comparative Analysis

arXiv 2605.23091 ↗First posted 21 May 2026cs.SE cs.AI cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

The majority of software developers use or are planning to use Artificial Intelligence (AI) tools in their development processes. Their top reasons include improving productivity and faster learning. In fact, Large Language Model (LLM)-generated code is currently in production, including in major tech companies. However, concerns were raised about the risks associated with the use of AI tools to generate code. In this paper, we focus our attention on the risks to software security. We empirically evaluate the security of code generated by seven popular LLMs. We build upon previous work to mimic the behaviours of developers when using LLMs to generate code. Our results show that all seven LLMs that we have evaluated generate code that contains vulnerabilities, the majority of which are of critical or high severity.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 77 references that are in the map. First posted on arXiv 21 May 2026.

Builds on

21 of 77 references.

References · 21

  1. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  2. 2022Do Users Write More Insecure Code with AI Assistants?cited by 186
  3. 2020You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completioncited by 112
  4. 2022Lost at C: A User Study on the Security Implications of Large Language Model Code Assistantscited by 87
  5. 2023How Secure is Code Generated by ChatGPT?cited by 77
  6. 2023GPTScan: Detecting Logic Vulnerabilities in Smart Contracts by Combining GPT with Program Analysiscited by 65
  7. 2022Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?cited by 63
  8. 2023LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluationscited by 58
  9. 2023A Survey on Large Language Models for Software Engineeringcited by 58
  10. 2023Prompt-Enhanced Software Vulnerability Detection Using ChatGPTcited by 51
  11. 2023TrojanPuzzle: Covertly Poisoning Code-Suggestion Modelscited by 47
  12. 2022An Empirical Study of Deep Learning Models for Vulnerability Detectioncited by 44
9 more
  1. 2024LeDex: Training LLMs to Better Self-Debug and Explain Codecited by 39
  2. 2024AI Hallucinations: A Misnomer Worth Clarifyingcited by 34
  3. 2024Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuningcited by 30
  4. 2023Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attackscited by 29
  5. 2023Assessing the Security of GitHub Copilot Generated Code -- A Targeted Replication Studycited by 24
  6. 2023Exploring ChatGPT's Capabilities on Vulnerability Managementcited by 19
  7. 2020SourceFinder: Finding Malware Source-Code from Publicly Available Repositoriescited by 14
  8. 2023Can Large Language Models Find And Fix Vulnerable Software?cited by 14
  9. 2025Guiding AI to Fix Its Own Flaws: An Empirical Study on LLM-Driven Secure Code Generationcited by 12

Built on by

Not cited yet.