mlsanityData to 7 Oct 2026

Prompt-Enhanced Software Vulnerability Detection Using ChatGPT

arXiv 2308.12697 ↗First posted 24 Aug 2023cs.SEcited by 51

Not on the map. Why?

Abstract (arXiv)

With the increase in software vulnerabilities that cause significant economic and social losses, automatic vulnerability detection has become essential in software development and maintenance. Recently, large language models (LLMs) like GPT have received considerable attention due to their stunning intelligence, and some studies consider using ChatGPT for vulnerability detection. However, they do not fully consider the characteristics of LLMs, since their designed questions to ChatGPT are simple without a specific prompt design tailored for vulnerability detection. This paper launches a study on the performance of software vulnerability detection using ChatGPT with different prompt designs. Firstly, we complement previous work by applying various improvements to the basic prompt. Moreover, we incorporate structural and sequential auxiliary information to improve the prompt design. Besides, we leverage ChatGPT's ability of memorizing multi-round dialogue to design suitable prompts for vulnerability detection. We conduct extensive experiments on two vulnerability datasets to demonstrate the effectiveness of prompt-enhanced vulnerability detection using ChatGPT. We also analyze the merit and demerit of using ChatGPT for vulnerability detection. Repository: https://github.com/KDEGroup/LLMVulnerabilityDetection.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Aug 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 21; 2025: 19; 2026: 8.

51 papers.

Sort:

By area:

  1. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  2. 2024Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAGcited by 67
  3. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 65
  4. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  5. 2024Agents in Software Engineering: Survey, Landscape, and Visioncited by 60
  6. 2024Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuningcited by 30
  7. 2023How Far Have We Gone in Vulnerability Detection Using Large Language Modelscited by 27
  8. 2024How secure is AI-generated Code: A Large-Scale Comparison of Large Language Modelscited by 23
  9. 2025KNighter: Transforming Static Analysis with LLM-Synthesized Checkerscited by 22
  10. 2025Let the Trial Begin: A Mock-Court Approach to Vulnerability Detection using LLM-Based Agentscited by 20
  11. 2024Multi-role Consensus through LLMs Discussions for Vulnerability Detectioncited by 19
  12. 2024To Err is Machine: Vulnerability Detection Challenges LLM Reasoningcited by 17
38 more
  1. 2024Harnessing Large Language Models for Software Vulnerability Detection: A Comprehensive Benchmarking Studycited by 17
  2. 2024MdEval: Massively Multilingual Code Debuggingcited by 16
  3. 2024LLbezpeky: Leveraging Large Language Models for Vulnerability Detectioncited by 13
  4. 2025A Contemporary Survey of Large Language Model Assisted Program Analysiscited by 10
  5. 2024From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Securitycited by 9
  6. 2024Software Vulnerability Prediction in Low-Resource Languages: An Empirical Study of CodeBERT and ChatGPTcited by 8
  7. 2025VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detectioncited by 8
  8. 2025Agent That Debugs: Dynamic State-Guided Vulnerability Repaircited by 7
  9. 2024An Insight into Security Code Review with LLMs: Capabilities, Obstacles, and Influential Factorscited by 6
  10. 2024CoDefeater: Using LLMs To Find Defeaters in Assurance Casescited by 5
  11. 2025Large Language Models for In-File Vulnerability Localization Can Be "Lost in the End"cited by 5
  12. 2025IFEvalCode: Controlled Code Generationcited by 5
  13. 2025Generative AI for Self-Adaptive Systems: State of the Art and Research Roadmapcited by 5
  14. 2025Improving LLM Reasoning for Vulnerability Detection via Group Relative Policy Optimizationcited by 4
  15. 2025RepoDebug: Repository-Level Multi-Task and Multi-Language Debugging Evaluation of Large Language Modelscited by 4
  16. 2025VulInstruct: Teaching LLMs Root-Cause Reasoning for Vulnerability Detection via Security Specificationscited by 4
  17. 2025A Systematic Study of Code Obfuscation Against LLM-based Vulnerability Detectioncited by 4
  18. 2026From SFT to RL: Demystifying the Post-Training Pipeline for LLM-based Vulnerability Detectioncited by 4
  19. 2025Evaluating LLaMA 3.2 for Software Vulnerability Detectioncited by 3
  20. 2025Revisiting Pre-trained Language Models for Vulnerability Detectioncited by 3
  21. 2025VULPO: Context-Aware Vulnerability Detection via On-Policy LLM Optimizationcited by 3
  22. 2024Generative transformations and patterns in LLM-native approaches for software verification and falsificationcited by 2
  23. 2024A Qualitative Study on Using ChatGPT for Software Security: Perception vs. Practicalitycited by 2
  24. 2025Identifying Helpful Context for LLM-based Vulnerability Repair: A Preliminary Studycited by 2
  25. 2023Exploring Large Language Models in Resolving Environment-Related Crash Bugs: Localizing and Repairingcited by 1
  26. 2024ANVIL: Anomaly-based Vulnerability Identification without Labelled Training Datacited by 1
  27. 2024Comparing Unidirectional, Bidirectional, and Word2vec Models for Discovering Vulnerabilities in Compiled Lifted Codecited by 1
  28. 2024FTSmartAudit: A Knowledge Distillation-Enhanced Framework for Automated Smart Contract Auditing Using Fine-Tuned LLMscited by 1
  29. 2024Direction for Detection: A Survey of Automated Vulnerability Detection and all of its Pain Pointscited by 0
  30. 2025Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniquescited by 0
  31. 2025WALL: A Web Application for Automated Quality Assurance using Large Language Modelscited by 0
  32. 2025C-ing Clearly: Enhanced Binary Code Explanations using C codecited by 0
  33. 2026Focus on What Matters: Fisher-Guided Adaptive Multimodal Fusion for Vulnerability Detectioncited by 0
  34. 2026One Model, Many Skills: Parameter-Efficient Fine-Tuning for Multitask Code Analysiscited by 0
  35. 2026Improving MPI Error Detection and Repair with Large Language Models and Bug Referencescited by 0
  36. 2026Security of LLM-generated Code: A Comparative Analysiscited by 0
  37. 2026Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detectioncited by 0
  38. 2026Evaluating Large Language Models for Symbolic Security Protocol Analysiscited by 0

50 most cited of 51 All 51 (JSON)