# Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilities

arXiv: https://arxiv.org/abs/2311.16169 · HTML: https://mlsanity.com/paper/2311.16169/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2023.

## Built on by (most cited first, 30 of 59)

- [Vulnerability Detection with Code Language Models: How Far Are We?](https://mlsanity.com/paper/2403.18624/) (2024)
- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG](https://mlsanity.com/paper/2406.11147/) (2024)
- [LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning](https://mlsanity.com/paper/2401.16185/) (2024)
- [LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights](https://mlsanity.com/paper/2502.07049/) (2025)
- [An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection](https://mlsanity.com/paper/2406.06822/) (2024)
- [How secure is AI-generated Code: A Large-Scale Comparison of Large Language Models](https://mlsanity.com/paper/2404.18353/) (2024)
- [Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis](https://mlsanity.com/paper/2502.01853/) (2025)
- [To Err is Machine: Vulnerability Detection Challenges LLM Reasoning](https://mlsanity.com/paper/2403.17218/) (2024)
- [Harnessing Large Language Models for Software Vulnerability Detection: A Comprehensive Benchmarking Study](https://mlsanity.com/paper/2405.15614/) (2024)
- [A Framework for Evaluating Emerging Cyberattack Capabilities of AI](https://mlsanity.com/paper/2503.11917/) (2025)
- [RTL-Breaker: Assessing the Security of LLMs against Backdoor Attacks on HDL Code Generation](https://mlsanity.com/paper/2411.17569/) (2024)
- [Measuring and Augmenting Large Language Models for Solving Capture-the-Flag Challenges](https://mlsanity.com/paper/2506.17644/) (2025)
- [A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code](https://mlsanity.com/paper/2508.18106/) (2025)
- [VulDetectBench: Evaluating the Deep Capability of Vulnerability Detection with Large Language Models](https://mlsanity.com/paper/2406.07595/) (2024)
- [From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Security](https://mlsanity.com/paper/2412.15004/) (2024)
- [Rethinking the Evaluation of Secure Code Generation](https://mlsanity.com/paper/2503.15554/) (2025)
- [VulnLLMEval: A Framework for Evaluating Large Language Models in Software Vulnerability Detection and Patching](https://mlsanity.com/paper/2409.10756/) (2024)
- [Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts](https://mlsanity.com/paper/2402.10691/) (2024)
- [SV-TrustEval-C: Evaluating Structure and Semantic Reasoning in Large Language Models for Source Code Vulnerability Analysis](https://mlsanity.com/paper/2505.20630/) (2025)
- [Reasoning with LLMs for Zero-Shot Vulnerability Detection](https://mlsanity.com/paper/2503.17885/) (2025)
- [An Insight into Security Code Review with LLMs: Capabilities, Obstacles, and Influential Factors](https://mlsanity.com/paper/2401.16310/) (2024)
- [ProveRAG: Provenance-Driven Vulnerability Analysis with Automated Retrieval-Augmented LLMs](https://mlsanity.com/paper/2410.17406/) (2024)
- [Asymmetric Bias in Text-to-Image Generation with Adversarial Attacks](https://mlsanity.com/paper/2312.14440/) (2023)
- [Trace Gadgets: Minimizing Code Context for Machine Learning-Based Vulnerability Prediction](https://mlsanity.com/paper/2504.13676/) (2025)
- [A Systematic Study of Code Obfuscation Against LLM-based Vulnerability Detection](https://mlsanity.com/paper/2512.16538/) (2025)
- [Reducing False Positives in Static Bug Detection with LLMs: An Empirical Study in Industry](https://mlsanity.com/paper/2601.18844/) (2026)
- [Sifting the Noise: A Comparative Study of LLM Agents in Vulnerability False Positive Filtering](https://mlsanity.com/paper/2601.22952/) (2026)
- [Can LLM Prompting Serve as a Proxy for Static Analysis in Vulnerability Detection](https://mlsanity.com/paper/2412.12039/) (2024)

## Abstract (arXiv)

While automated vulnerability detection techniques have made promising progress in detecting security vulnerabilities, their scalability and applicability remain challenging. The remarkable performance of Large Language Models (LLMs), such as GPT-4 and CodeLlama, on code-related tasks has prompted recent works to explore if LLMs can be used to detect vulnerabilities. In this paper, we perform a more comprehensive study by concurrently examining a higher number of datasets, languages and LLMs, and qualitatively evaluating performance across prompts and vulnerability classes while addressing the shortcomings of existing tools. Concretely, we evaluate the effectiveness of 16 pre-trained LLMs on 5,000 code samples from five diverse security datasets. These balanced datasets encompass both synthetic and real-world projects in Java and C/C++ and cover 25 distinct vulnerability classes. Overall, LLMs across all scales and families show modest effectiveness in detecting vulnerabilities, obtaining an average accuracy of 62.8% and F1 score of 0.71 across datasets. They are significantly better at detecting vulnerabilities only requiring intra-procedural analysis, such as OS Command Injection and NULL Pointer Dereference. Moreover, they report higher accuracies on these vulnerabilities than popular static analysis tools, such as CodeQL. We find that advanced prompting strategies that involve step-by-step analysis significantly improve performance of LLMs on real-world datasets in terms of F1 score (by upto 0.18 on average). Interestingly, we observe that LLMs show promising abilities at performing parts of the analysis correctly, such as identifying vulnerability-related specifications and leveraging natural language information to understand code behavior (e.g., to check if code is sanitized). We expect our insights to guide future work on LLM-augmented vulnerability detection systems.