mlsanityData to 14 Oct 2026

Security in LLM-Assisted Coding: A Systematic Literature Review

arXiv 2610.15482 ↗First posted 11 Oct 2026cs.CR cs.AI cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models accelerate software development but introduce severe security vulnerabilities. This systematic review investigates security risks in automated coding, benchmark limitations, and the role of human developers. Findings indicate that models frequently generate insecure code and faulty patches due to training data biases. Additionally, the models themselves are susceptible to direct attacks. Current evaluation methods suffer from data contamination and lack robust security metrics. Consequently, human oversight remains essential. Safer integration of automation will require more security-aware model architectures, stronger security-specific benchmarks, and more effective human-AI collaboration frameworks to mitigate these systemic risks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 71 of its 136 references that are in the map. First posted on arXiv 11 Oct 2026.

Builds on

71 of 136 references.

References · 71

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  6. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  7. 2021Measuring Coding Challenge Competence With APPScited by 902
  8. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 479
  9. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
  10. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  11. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 300
  12. 2019Adversarial Examples Improve Image Recognitioncited by 286
38 more
  1. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 247
  2. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 218
  3. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 197
  4. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  5. 2022Do Users Write More Insecure Code with AI Assistants?cited by 192
  6. 2023ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generationcited by 168
  7. 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 155
  8. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 138
  9. 2020You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completioncited by 131
  10. 2022Lost at C: A User Study on the Security Implications of Large Language Model Code Assistantscited by 91
  11. 2023How Secure is Code Generated by ChatGPT?cited by 81
  12. 2023Improving Code Generation by Training with Natural Language Feedbackcited by 76
  13. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 76
  14. 2023Can ChatGPT replace StackOverflow? A Study on Robustness and Reliability of Large Language Model Code Generationcited by 73
  15. 2024Reinforcement Learning Enhanced LLMs: A Surveycited by 71
  16. 2024Instruction Tuning for Secure Code Generationcited by 68
  17. 2022Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?cited by 67
  18. 2022CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Contextcited by 65
  19. 2023LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluationscited by 62
  20. 2023TrojanPuzzle: Covertly Poisoning Code-Suggestion Modelscited by 60
  21. 2025CWEval: Outcome-driven Evaluation on Functionality and Security of LLM Code Generationcited by 59
  22. 2021AdvDrop: Adversarial Attack to DNNs by Dropping Informationcited by 56
  23. 2023CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Modelscited by 55
  24. 2024What's Wrong with Your Code Generated by Large Language Models? An Extensive Studycited by 53
  25. 2023No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPTcited by 47
  26. 2025LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insightscited by 47
  27. 2024An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detectioncited by 40
  28. 2023SALLM: Security Assessment of Generated Codecited by 39
  29. 2024Constrained Decoding for Secure Code Generationcited by 37
  30. 2024Automatic Programming: Large Language Models and Beyondcited by 36
  31. 2024AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testingcited by 35
  32. 2023The FormAI Dataset: Generative AI in Software Security Through the Lens of Formal Verificationcited by 34
  33. 2024Test-Driven Development for Code Generationcited by 34
  34. 2023Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attackscited by 33
  35. 2020Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Modelscited by 28
  36. 2024HumanEvo: An Evolution-aware Benchmark for More Realistic Evaluation of Repository-level Code Generationcited by 28
  37. 2024LLMs in Web Development: Evaluating LLM-Generated PHP Code Unveiling Vulnerabilities and Limitationscited by 22
  38. 2024Can We Trust Large Language Models Generated Code? A Framework for In-Context Learning, Security Patterns, and Code Evaluations Across Diverse LLMscited by 19

50 most cited of 71 All 71 (JSON)

Built on by

Not cited yet.