mlsanityData to 7 Oct 2026

How Secure is Code Generated by ChatGPT?

arXiv 2304.09655 ↗First posted 19 Apr 2023cs.CRcited by 77

Not on the map. Why?

Abstract (arXiv)

In recent years, large language models have been responsible for great advances in the field of artificial intelligence (AI). ChatGPT in particular, an AI chatbot developed and recently released by OpenAI, has taken the field to the next level. The conversational model is able not only to process human-like text, but also to translate natural language into code. However, the safety of programs generated by ChatGPT should not be overlooked. In this paper, we perform an experiment to address this issue. Specifically, we ask ChatGPT to generate a number of program and evaluate the security of the resulting source code. We further investigate whether ChatGPT can be prodded to improve the security by appropriate prompts, and discuss the ethical aspects of using AI to generate code. Results suggest that ChatGPT is aware of potential vulnerabilities, but nonetheless often generates source code that are not robust to certain attacks.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Apr 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 26; 2025: 27; 2026: 16.

77 papers.

Sort:

By area:

  1. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 130
  2. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  3. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  4. 2024Instruction Tuning for Secure Code Generationcited by 63
  5. 2023Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Taskscited by 59
  6. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 36
  7. 2024Constrained Decoding for Secure Code Generationcited by 35
  8. 2024AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testingcited by 34
  9. 2023The FormAI Dataset: Generative AI in Software Security Through the Lens of Formal Verificationcited by 32
  10. 2024Toward Secure Code Generation: Bridging Correctness and Security via Task-Adaptive Vulnerability Modeling and Execution-Based Benchmarkingcited by 26
  11. 2024Security of AI Agentscited by 24
  12. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
38 more
  1. 2024How secure is AI-generated Code: A Large-Scale Comparison of Large Language Modelscited by 23
  2. 2025Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projectscited by 23
  3. 2023LLM for SoC Security: A Paradigm Shiftcited by 22
  4. 2025Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysiscited by 18
  5. 2025Correctness Assessment of Code Generated by Large Language Models Using Internal Representationscited by 17
  6. 2024RePair: Automated Program Repair with Process-based Feedbackcited by 16
  7. 2023Efficient Avoidance of Vulnerabilities in Auto-completed Smart Contract Code Using Vulnerability-constrained Decodingcited by 14
  8. 2024HexaCoder: Secure Code Generation via Oracle-Guided Synthetic Training Datacited by 14
  9. 2024Black-Box Adversarial Attacks on LLM-Based Code Completioncited by 13
  10. 2025Benchmarking Prompt Engineering Techniques for Secure Code Generation with GPT Modelscited by 12
  11. 2024An Investigation into Misuse of Java Security APIs by Large Language Modelscited by 11
  12. 2024Code Security Vulnerability Repair Using Reinforcement Learning with Large Language Modelscited by 10
  13. 2025SandboxEval: Towards Securing Test Environment for Untrusted Codecited by 10
  14. 2025Teaching an Old LLM Secure Coding: Localized Preference Optimization on Distilled Preferencescited by 10
  15. 2024From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Securitycited by 9
  16. 2025Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generationcited by 9
  17. 2025Hallucination by Code Generation LLMs: Taxonomy, Benchmarks, Mitigation, and Challengescited by 9
  18. 2025I Know Which LLM Wrote Your Code Last Summer: LLM generated Code Stylometry for Authorship Attributioncited by 9
  19. 2025Do LLMs Consider Security? An Empirical Study on Responses to Programming Questionscited by 8
  20. 2024A Controlled Experiment on the Energy Efficiency of the Source Code Generated by Code Llamacited by 7
  21. 2024INDICT: Code Generation with Internal Dialogues of Critiques for Both Security and Helpfulnesscited by 7
  22. 2024How Well Do Large Language Models Serve as End-to-End Secure Code Agents for Python?cited by 7
  23. 2025The Hidden Risks of LLM-Generated Web Application Code: A Security-Centric Evaluation of Code Generation Capabilities in Large Language Modelscited by 7
  24. 2025CodeMirage: A Multi-Lingual Benchmark for Detecting AI-Generated and Paraphrased Source Code from Production-Level LLMscited by 6
  25. 2023Exploring the Limits of ChatGPT in Software Security Applicationscited by 5
  26. 2024Code Hallucinationcited by 5
  27. 2025Achilles Heel of Distributed Multi-Agent Systemscited by 5
  28. 2025How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-benchcited by 5
  29. 2023LUNA: A Model-Based Universal Analysis Framework for Large Language Modelscited by 4
  30. 2024Stars, Stripes, and Silicon: Unravelling the ChatGPT's All-American, Monochrome, Cis-centric Biascited by 4
  31. 2025Automated Consistency Analysis of LLMscited by 4
  32. 2025Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigmcited by 4
  33. 2024CodeCloak: A Method for Evaluating and Mitigating Code Leakage by LLM Code Assistantscited by 3
  34. 2024"You still have to study" -- On the Security of LLM generated codecited by 3
  35. 2024Can LLM Prompting Serve as a Proxy for Static Analysis in Vulnerability Detectioncited by 3
  36. 2025Is Your Prompt Poisoning Code? Defect Induction Rates and Security Mitigation Strategiescited by 3
  37. 2026The End of Code Review: Coding Agents Supersede Human Inspectioncited by 3
  38. 2024A Comprehensive Framework for Evaluating API-oriented Code Generation in Large Language Modelscited by 2

50 most cited of 77 All 77 (JSON)