# Evaluation of ChatGPT Model for Vulnerability Detection

arXiv: https://arxiv.org/abs/2304.07232 · HTML: https://mlsanity.com/paper/2304.07232/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 12 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 6 of 6)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models](https://mlsanity.com/paper/2303.10420/) (2023)
- [GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://mlsanity.com/paper/2104.08826/) (2021)

## Built on by (most cited first, 30 of 33)

- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets](https://mlsanity.com/paper/2305.18486/) (2023)
- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems](https://mlsanity.com/paper/2402.18649/) (2024)
- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [GPTScan: Detecting Logic Vulnerabilities in Smart Contracts by Combining GPT with Program Analysis](https://mlsanity.com/paper/2308.03314/) (2023)
- [Finetuning Large Language Models for Vulnerability Detection](https://mlsanity.com/paper/2401.17010/) (2024)
- [DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructions](https://mlsanity.com/paper/2312.04730/) (2023)
- [Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning](https://mlsanity.com/paper/2406.03718/) (2024)
- [How Far Have We Gone in Vulnerability Detection Using Large Language Models](https://mlsanity.com/paper/2311.12420/) (2023)
- [Harnessing Large Language Models for Software Vulnerability Detection: A Comprehensive Benchmarking Study](https://mlsanity.com/paper/2405.15614/) (2024)
- [Can Large Language Models Find And Fix Vulnerable Software?](https://mlsanity.com/paper/2308.10345/) (2023)
- [Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models](https://mlsanity.com/paper/2407.00456/) (2024)
- [Evaluation of LLM Chatbots for OSINT-based Cyber Threat Awareness](https://mlsanity.com/paper/2401.15127/) (2024)
- [LLbezpeky: Leveraging Large Language Models for Vulnerability Detection](https://mlsanity.com/paper/2401.01269/) (2024)
- [GenKubeSec: LLM-Based Kubernetes Misconfiguration Detection, Localization, Reasoning, and Remediation](https://mlsanity.com/paper/2405.19954/) (2024)
- [Software Vulnerability Prediction in Low-Resource Languages: An Empirical Study of CodeBERT and ChatGPT](https://mlsanity.com/paper/2404.17110/) (2024)
- [Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions](https://mlsanity.com/paper/2502.14202/) (2025)
- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [A Preliminary Analysis on the Code Generation Capabilities of GPT-3.5 and Bard AI Models for Java Functions](https://mlsanity.com/paper/2305.09402/) (2023)
- [Using ChatGPT as a Static Application Security Testing Tool](https://mlsanity.com/paper/2308.14434/) (2023)
- [Agent That Debugs: Dynamic State-Guided Vulnerability Repair](https://mlsanity.com/paper/2504.07634/) (2025)
- [Exploring the Limits of ChatGPT in Software Security Applications](https://mlsanity.com/paper/2312.05275/) (2023)
- [ProveRAG: Provenance-Driven Vulnerability Analysis with Automated Retrieval-Augmented LLMs](https://mlsanity.com/paper/2410.17406/) (2024)
- [Defining and Detecting Vulnerability in Human Evaluation Guidelines: A Preliminary Study Towards Reliable NLG Evaluation](https://mlsanity.com/paper/2406.07935/) (2024)
- [Large Language Models for In-File Vulnerability Localization Can Be "Lost in the End"](https://mlsanity.com/paper/2502.06898/) (2025)
- [How Does Naming Affect LLMs on Code Analysis Tasks?](https://mlsanity.com/paper/2307.12488/) (2023)
- [ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generation](https://mlsanity.com/paper/2601.09703/) (2026)
- [Evaluating GPT's Programming Capability through CodeWars' Katas](https://mlsanity.com/paper/2306.01784/) (2023)
- [LLM Embedding-based Attribution (LEA): Quantifying Source Contributions to Generative Model's Response for Vulnerability Analysis](https://mlsanity.com/paper/2506.12100/) (2025)

## Abstract (arXiv)

In this technical report, we evaluated the performance of the ChatGPT and GPT-3 models for the task of vulnerability detection in code. Our evaluation was conducted on our real-world dataset, using binary and multi-label classification tasks on CWE vulnerabilities. We decided to evaluate the model because it has shown good performance on other code-based tasks, such as solving programming challenges and understanding code at a high level. However, we found that the ChatGPT model performed no better than a dummy classifier for both binary and multi-label classification tasks for code vulnerability detection.