# Chain-of-Thought Prompting of Large Language Models for Discovering and Fixing Software Vulnerabilities

arXiv: https://arxiv.org/abs/2402.17230 · HTML: https://mlsanity.com/paper/2402.17230/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2024.

## Built on by (most cited first, 30 of 31)

- [Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG](https://mlsanity.com/paper/2406.11147/) (2024)
- [Multi-role Consensus through LLMs Discussions for Vulnerability Detection](https://mlsanity.com/paper/2403.14274/) (2024)
- [To Err is Machine: Vulnerability Detection Challenges LLM Reasoning](https://mlsanity.com/paper/2403.17218/) (2024)
- [Harnessing Large Language Models for Software Vulnerability Detection: A Comprehensive Benchmarking Study](https://mlsanity.com/paper/2405.15614/) (2024)
- [Investigating Large Language Models for Code Vulnerability Detection: An Experimental Study](https://mlsanity.com/paper/2412.18260/) (2024)
- [From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Security](https://mlsanity.com/paper/2412.15004/) (2024)
- [VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection](https://mlsanity.com/paper/2509.11523/) (2025)
- [R2Vul: Learning to Reason about Software Vulnerabilities with Reinforcement Learning and Structured Reasoning Distillation](https://mlsanity.com/paper/2504.04699/) (2025)
- [Can LLMs Handle WebShell Detection? Overcoming Detection Challenges with Behavioral Function-Aware Framework](https://mlsanity.com/paper/2504.13811/) (2025)
- [MAVUL: Multi-Agent Vulnerability Detection via Contextual Reasoning and Interactive Refinement](https://mlsanity.com/paper/2510.00317/) (2025)
- [Co-RedTeam: Orchestrated Security Discovery and Exploitation with LLM Agents](https://mlsanity.com/paper/2602.02164/) (2026)
- [AutoPatch: Multi-Agent Framework for Patching Real-World CVE Vulnerabilities](https://mlsanity.com/paper/2505.04195/) (2025)
- [VulInstruct: Teaching LLMs Root-Cause Reasoning for Vulnerability Detection via Security Specifications](https://mlsanity.com/paper/2511.04014/) (2025)
- [CoTDeceptor:Adversarial Code Obfuscation Against CoT-Enhanced LLM Code Agents](https://mlsanity.com/paper/2512.21250/) (2025)
- [DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection](https://mlsanity.com/paper/2607.13439/) (2026)
- [Generative transformations and patterns in LLM-native approaches for software verification and falsification](https://mlsanity.com/paper/2404.09384/) (2024)
- [A Qualitative Study on Using ChatGPT for Software Security: Perception vs. Practicality](https://mlsanity.com/paper/2408.00435/) (2024)
- [XiHeFusion: Harnessing Large Language Models for Science Communication in Nuclear Fusion](https://mlsanity.com/paper/2502.05615/) (2025)
- [Llama-based source code vulnerability detection: Prompt engineering vs Fine tuning](https://mlsanity.com/paper/2512.09006/) (2025)
- [PromptAudit: Auditing Prompt Sensitivity in LLM-Based Vulnerability Detection](https://mlsanity.com/paper/2605.24171/) (2026)
- [Enhancing Reliability in LLM-Based Secure Code Generation](https://mlsanity.com/paper/2605.24300/) (2026)
- [Fine-Tuning LLMs to Analyze Multiple Dimensions of Code Review: A Maximum Entropy Regulated Long Chain-of-Thought Approach](https://mlsanity.com/paper/2509.21170/) (2025)
- [Three Heads Are Better Than One: A Multi-perspective Reasoning Framework for Enhanced Vulnerability Detection](https://mlsanity.com/paper/2605.18153/) (2026)
- [ReVul-CoT: Towards Effective Software Vulnerability Assessment with Retrieval-Augmented Generation and Chain-of-Thought Prompting](https://mlsanity.com/paper/2511.17027/) (2025)
- [SoK: Understanding (New) Security Issues Across AI4Code Use Cases](https://mlsanity.com/paper/2512.18456/) (2025)
- [A Quasi-Experimental Developer Study of Security Training in LLM-Assisted Web Application Development](https://mlsanity.com/paper/2604.17763/) (2026)
- [Generating Place-Based Compromises Between Two Points of View](https://mlsanity.com/paper/2604.24536/) (2026)
- [Representation Matters: An Empirical Study of Program Representations for LLM Vulnerability Reasoning](https://mlsanity.com/paper/2606.25356/) (2026)
- [Evaluating Large Language Models for Symbolic Security Protocol Analysis](https://mlsanity.com/paper/2607.20712/) (2026)
- [Reveree: Diagnosing LLM Reverse-Engineering Agents](https://mlsanity.com/paper/2609.01185/) (2026)

## Abstract (arXiv)

Security vulnerabilities are increasingly prevalent in modern software and they are widely consequential to our society. Various approaches to defending against these vulnerabilities have been proposed, among which those leveraging deep learning (DL) avoid major barriers with other techniques hence attracting more attention in recent years. However, DL-based approaches face critical challenges including the lack of sizable and quality-labeled task-specific datasets and their inability to generalize well to unseen, real-world scenarios. Lately, large language models (LLMs) have demonstrated impressive potential in various domains by overcoming those challenges, especially through chain-of-thought (CoT) prompting. In this paper, we explore how to leverage LLMs and CoT to address three key software vulnerability analysis tasks: identifying a given type of vulnerabilities, discovering vulnerabilities of any type, and patching detected vulnerabilities. We instantiate the general CoT methodology in the context of these tasks through VSP , our unified, vulnerability-semantics-guided prompting approach, and conduct extensive experiments assessing VSP versus five baselines for the three tasks against three LLMs and two datasets. Results show substantial superiority of our CoT-inspired prompting (553.3%, 36.5%, and 30.8% higher F1 accuracy for vulnerability identification, discovery, and patching, respectively, on CVE datasets) over the baselines. Through in-depth case studies analyzing VSP failures, we also reveal current gaps in LLM/CoT for challenging vulnerability cases, while proposing and validating respective improvements.