mlsanityData to 13 Oct 2026

Using ChatGPT as a Static Application Security Testing Tool

arXiv 2308.14434 ↗First posted 28 Aug 2023cs.CR cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

In recent years, artificial intelligence has had a conspicuous growth in almost every aspect of life. One of the most applicable areas is security code review, in which a lot of AI-based tools and approaches have been proposed. Recently, ChatGPT has caught a huge amount of attention with its remarkable performance in following instructions and providing a detailed response. Regarding the similarities between natural language and code, in this paper, we study the feasibility of using ChatGPT for vulnerability detection in Python source code. Toward this goal, we feed an appropriate prompt along with vulnerable data to ChatGPT and compare its results on two datasets with the results of three widely used Static Application Security Testing tools (Bandit, Semgrep and SonarQube). We implement different kinds of experiments with ChatGPT and the results indicate that ChatGPT reduces the false positive and false negative rates and has the potential to be used for Python source code vulnerability detection.

Key facts. Cited by 8 papers in the map (data through 13 Oct 2026). Builds on 11 of its 32 references that are in the map. First posted on arXiv 28 Aug 2023.

Builds on

11 of 32 references.

References · 11

  1. 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 272
  2. 2020Deep Learning based Vulnerability Detection: Are We There Yet?cited by 144
  3. 2018Automated Vulnerability Detection in Source Code Using Deep Representation Learningcited by 114
  4. 2023DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detectioncited by 101
  5. 2023LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluationscited by 61
  6. 2022VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detectioncited by 51
  7. 2022An Empirical Study of Deep Learning Models for Vulnerability Detectioncited by 45
  8. 2023Evaluation of ChatGPT Model for Vulnerability Detectioncited by 33
  9. 2021VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatementscited by 20
  10. 2022VUDENC: Vulnerability Detection with Deep Learning on a Natural Codebase for Pythoncited by 17
  11. 2022Graph Neural Networks: a bibliometrics overviewcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 5; 2025: 2.

8 papers.

  1. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  2. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 71
  3. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 68
  4. 2024From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Securitycited by 9
  5. 2025Do LLMs Consider Security? An Empirical Study on Responses to Programming Questionscited by 9
  6. 2024An Insight into Security Code Review with LLMs: Capabilities, Obstacles, and Influential Factorscited by 6
  7. 2024CodeCloak: A Method for Evaluating and Mitigating Code Leakage by LLM Code Assistantscited by 3
  8. 2025Towards Effective Complementary Security Analysis using Large Language Modelscited by 1