LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) are emerging as transformative tools for software vulnerability detection, addressing critical challenges in the security domain. Traditional methods, such as static and dynamic analysis, often falter due to inefficiencies, high false positive rates, and the growing complexity of modern software systems. By leveraging their ability to analyze code structures, identify patterns, and generate repair suggestions, LLMs, exemplified by models like GPT, BERT, and CodeBERT, present a novel and scalable approach to mitigating vulnerabilities. This paper provides a detailed survey of LLMs in vulnerability detection. It examines key aspects, including model architectures, application methods, target languages, fine-tuning strategies, datasets, and evaluation metrics. We also analyze the scope of current research problems, highlighting the strengths and weaknesses of existing approaches. Further, we address challenges such as cross-language vulnerability detection, multimodal data integration, and repository-level analysis. Based on these findings, we propose solutions for issues like dataset scalability, model interpretability, and applications in low-resource scenarios. Our contributions are threefold: (1) a systematic review of how LLMs are applied in vulnerability detection; (2) an analysis of shared patterns and differences across studies, with a unified framework for understanding the field; and (3) a summary of key challenges and future research directions. This work provides valuable insights for advancing LLM-based vulnerability detection. We also maintain and regularly update latest selected paper on https://github.com/OwenSanzas/LLM-For-Vulnerability-Detection
Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 73 of its 135 references that are in the map. First posted on arXiv 10 Feb 2025.
Builds on
73 of 135 references.
References · 73
- 2017Attention Is All You Needcited by 59,932
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
- 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
- 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
- 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
- 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
38 more
- 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
- 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
- 2021Unified Pre-training for Program Understanding and Generationcited by 361
- 2022A Systematic Evaluation of Large Language Models of Codecited by 360
- 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 355
- 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 266
- 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 149
- 2024Vulnerability Detection with Code Language Models: How Far Are We?cited by 133
- 2023AugGPT: Leveraging ChatGPT for Text Data Augmentationcited by 128
- 2024LLM Agents can Autonomously Exploit One-day Vulnerabilitiescited by 125
- 2024IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilitiescited by 100
- 2023DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detectioncited by 98
- 2023LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarkscited by 94
- 2018SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilitiescited by 91
- 2021CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Softwarecited by 89
- 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
- 2023Large Language Models for Test-Free Fault Localizationcited by 67
- 2024Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAGcited by 67
- 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
- 2023Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilitiescited by 58
- 2022NatGen: Generative pre-training by "Naturalizing" source codecited by 53
- 2022VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detectioncited by 51
- 2023How Effective Are Neural Networks for Fixing Security Vulnerabilitiescited by 50
- 2021D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysiscited by 48
- 2023Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectivescited by 48
- 2023ChatGPT for Vulnerability Detection, Classification, and Repair: How Far Are We?cited by 44
- 2024Combining Fine-Tuning and LLM-based Agents for Intuitive Smart Contract Auditing with Justificationscited by 41
- 2024Finetuning Large Language Models for Vulnerability Detectioncited by 39
- 2020VulDeeLocator: A Deep Learning-based Fine-grained Vulnerability Detectorcited by 37
- 2024Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuningcited by 30
- 2019A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Softwarecited by 28
- 2023Prompt Fuzzing for Fuzz Driver Generationcited by 27
- 2024How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queriescited by 26
- 2023Nuances are the Key: Unlocking ChatGPT to Find Failure-Inducing Tests with Differential Promptingcited by 23
- 2024Multitask-based Evaluation of Open-Source LLM on Software Vulnerabilitycited by 23
- 2023How Effective Are They? Exploring Large Language Model Based Fuzz Driver Generationcited by 21
- 2020How Effective are Smart Contract Analysis Tools? Evaluating Smart Contract Static Analysis Tools Using Bug Injectioncited by 20
- 2024Multi-role Consensus through LLMs Discussions for Vulnerability Detectioncited by 19
50 most cited of 73 All 73 (JSON)
Built on by
Citing papers per year: 2025: 11; 2026: 33.
44 papers.
Sort:
By area:
- 2025VulnLLM-R: Specialized Reasoning LLM with Agent Scaffold for Vulnerability Detectioncited by 11
- 2025Large Language Models for Security Operations Centers: A Comprehensive Surveycited by 9
- 2025A Systematic Literature Review on Detecting Software Vulnerabilities with Large Language Modelscited by 8
- 2026MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolutioncited by 6
- 2025How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-benchcited by 5
- 2026Reducing False Positives in Static Bug Detection with LLMs: An Empirical Study in Industrycited by 5
- 2025AutoPatch: Multi-Agent Framework for Patching Real-World CVE Vulnerabilitiescited by 4
- 2025Agentic Discovery and Validation of Android App Vulnerabilitiescited by 4
- 2026CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarkingcited by 3
- 2026Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Reviewcited by 3
- 2026FuzzingBrain V2: A Multi-Agent LLM System for Automated Vulnerability Discovery and Reproductioncited by 3
- 2026DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detectioncited by 3
32 more
- 2025Data and Context Matter: Towards Generalizing AI-based Software Vulnerability Detectioncited by 2
- 2025Unified Threat Detection and Mitigation Framework (UTDMF): Combating Prompt Injection, Deception, and Bias in Enterprise-Scale Transformerscited by 1
- 2025Securing the AI Supply Chain: What Can We Learn From Developer-Reported Security Issues and Solutions of AI Projects?cited by 1
- 2026A Dual-Loop Agent Framework for Automated Vulnerability Reproductioncited by 1
- 2026Red-MIRROR: Agentic LLM-based Autonomous Penetration Testing with Reflective Verification and Knowledge-augmented Interactioncited by 1
- 2026Three Heads Are Better Than One: A Multi-perspective Reasoning Framework for Enhanced Vulnerability Detectioncited by 1
- 2026Refusal Evaluation in Coding LLMs and Code Agents: A Systematic Review of Thirteen Malicious-Code Prompt Corpora (2023-2025)cited by 1
- 2026Agora: Toward Autonomous Bug Detection in Production-Level Consensus Protocols with LLM Agentscited by 1
- 2026Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implicationscited by 1
- 2026RustMizan: A Compilable, Contamination-Aware Benchmarking Framework for Rust Vulnerabilitiescited by 1
- 2026EntailLLM: Verifying LLM-Generated Vulnerability Discovery Paths with Domain Knowledge via Logic Programmingcited by 1
- 2025On the Effectiveness of Instruction-Tuning Local LLMs for Identifying Software Vulnerabilitiescited by 0
- 2025Assessing the Software Security Comprehension of Large Language Modelscited by 0
- 2026RiskBridge: Turning CVEs into Business-Aligned Patch Prioritiescited by 0
- 2026Seclens: Role-specific Evaluation of LLM's for security vulnerablity detectioncited by 0
- 2026Improving MPI Error Detection and Repair with Large Language Models and Bug Referencescited by 0
- 2026Learning Generalizable Multimodal Representations for Software Vulnerability Detectioncited by 0
- 2026One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intentcited by 0
- 2026Willing but Unable: Separating Refusal from Capability in Code LLMs via Abliterationcited by 0
- 2026Can Open-Source LLM Agents Replace Static Application Security Testing Tools? An Empirical Assessmentcited by 0
- 2026Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detectioncited by 0
- 2026Understanding Implicit Trust Errors in Core Carrier Networks through Multi-Agent Flaw Discovery and Analysiscited by 0
- 2026From Adoption to Deployment: A Qualitative Study on AI Integration in Software Development Practicecited by 0
- 2026Evaluating Large Language Models for Symbolic Security Protocol Analysiscited by 0
- 2026ALIBI: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors via Adversarial Code Commentscited by 0
- 2026CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detectioncited by 0
- 2026Cross-Corpus Evaluation of Generalizable Vulnerability Detection in IoT Firmwarecited by 0
- 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automationcited by 0
- 2026Large Language Models at the Intersection of Software Engineering and Software Security:An Evidence-Centered Structured Survey and Research Agendacited by 0
- 2026FuzzingBrain-Bench V1: Evaluating Open-Ended Bug Discovery by LLMscited by 0
- 2026Automated Vulnerability Injection in Smart Contracts Using Large Language Modelscited by 0
- 2026Prompt Injection Detection for Email Agents Through Attack Chain Modelingcited by 0
- 2026Prompt Injection Detection for Email Agents Through Attack Chain Modeling25 Sep 2026
- 2026Automated Vulnerability Injection in Smart Contracts Using Large Language Models2 Sep 2026
- 2026FuzzingBrain-Bench V1: Evaluating Open-Ended Bug Discovery by LLMs25 Aug 2026
- 2026Large Language Models at the Intersection of Software Engineering and Software Security:An Evidence-Centered Structured Survey and Research Agenda21 Aug 2026
- 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automation18 Aug 2026
- 2026Cross-Corpus Evaluation of Generalizable Vulnerability Detection in IoT Firmware11 Aug 2026
- 2026CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detection4 Aug 2026
- 2026EntailLLM: Verifying LLM-Generated Vulnerability Discovery Paths with Domain Knowledge via Logic Programming3 Aug 2026
- 2026ALIBI: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors via Adversarial Code Comments27 Jul 2026
- 2026Evaluating Large Language Models for Symbolic Security Protocol Analysis22 Jul 2026
- 2026From Adoption to Deployment: A Qualitative Study on AI Integration in Software Development Practice18 Jul 2026
- 2026DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection15 Jul 2026
32 more
- 2026Understanding Implicit Trust Errors in Core Carrier Networks through Multi-Agent Flaw Discovery and Analysis11 Jul 2026
- 2026RustMizan: A Compilable, Contamination-Aware Benchmarking Framework for Rust Vulnerabilities6 Jul 2026
- 2026Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection29 Jun 2026
- 2026Can Open-Source LLM Agents Replace Static Application Security Testing Tools? An Empirical Assessment10 Jun 2026
- 2026Willing but Unable: Separating Refusal from Capability in Code LLMs via Abliteration3 Jun 2026
- 2026Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implications3 Jun 2026
- 2026Agora: Toward Autonomous Bug Detection in Production-Level Consensus Protocols with LLM Agents28 May 2026
- 2026FuzzingBrain V2: A Multi-Agent LLM System for Automated Vulnerability Discovery and Reproduction20 May 2026
- 2026Refusal Evaluation in Coding LLMs and Code Agents: A Systematic Review of Thirteen Malicious-Code Prompt Corpora (2023-2025)19 May 2026
- 2026Three Heads Are Better Than One: A Multi-perspective Reasoning Framework for Enhanced Vulnerability Detection18 May 2026
- 2026One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent7 May 2026
- 2026Learning Generalizable Multimodal Representations for Software Vulnerability Detection28 Apr 2026
- 2026Improving MPI Error Detection and Repair with Large Language Models and Bug References2 Apr 2026
- 2026Seclens: Role-specific Evaluation of LLM's for security vulnerablity detection2 Apr 2026
- 2026Red-MIRROR: Agentic LLM-based Autonomous Penetration Testing with Reflective Verification and Knowledge-augmented Interaction28 Mar 2026
- 2026Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Review19 Mar 2026
- 2026CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarking8 Feb 2026
- 2026A Dual-Loop Agent Framework for Automated Vulnerability Reproduction5 Feb 2026
- 2026MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution26 Jan 2026
- 2026Reducing False Positives in Static Bug Detection with LLMs: An Empirical Study in Industry26 Jan 2026
- 2026RiskBridge: Turning CVEs into Business-Aligned Patch Priorities8 Jan 2026
- 2025Securing the AI Supply Chain: What Can We Learn From Developer-Reported Security Issues and Solutions of AI Projects?29 Dec 2025
- 2025Assessing the Software Security Comprehension of Large Language Models24 Dec 2025
- 2025On the Effectiveness of Instruction-Tuning Local LLMs for Identifying Software Vulnerabilities23 Dec 2025
- 2025VulnLLM-R: Specialized Reasoning LLM with Agent Scaffold for Vulnerability Detection8 Dec 2025
- 2025Unified Threat Detection and Mitigation Framework (UTDMF): Combating Prompt Injection, Deception, and Bias in Enterprise-Scale Transformers6 Oct 2025
- 2025Large Language Models for Security Operations Centers: A Comprehensive Survey13 Sep 2025
- 2025Agentic Discovery and Validation of Android App Vulnerabilities29 Aug 2025
- 2025Data and Context Matter: Towards Generalizing AI-based Software Vulnerability Detection14 Aug 2025
- 2025A Systematic Literature Review on Detecting Software Vulnerabilities with Large Language Models30 Jul 2025
- 2025How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench30 Jun 2025
- 2025AutoPatch: Multi-Agent Framework for Patching Real-World CVE Vulnerabilities7 May 2025
- 2025How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-benchcited by 5
- 2025AutoPatch: Multi-Agent Framework for Patching Real-World CVE Vulnerabilitiescited by 4
- 2025Agentic Discovery and Validation of Android App Vulnerabilitiescited by 4
- 2026CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarkingcited by 3
- 2026FuzzingBrain V2: A Multi-Agent LLM System for Automated Vulnerability Discovery and Reproductioncited by 3
- 2026A Dual-Loop Agent Framework for Automated Vulnerability Reproductioncited by 1
- 2026Red-MIRROR: Agentic LLM-based Autonomous Penetration Testing with Reflective Verification and Knowledge-augmented Interactioncited by 1
- 2026Refusal Evaluation in Coding LLMs and Code Agents: A Systematic Review of Thirteen Malicious-Code Prompt Corpora (2023-2025)cited by 1
- 2026Agora: Toward Autonomous Bug Detection in Production-Level Consensus Protocols with LLM Agentscited by 1
- 2026Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implicationscited by 1
- 2026RustMizan: A Compilable, Contamination-Aware Benchmarking Framework for Rust Vulnerabilitiescited by 1
- 2026Can Open-Source LLM Agents Replace Static Application Security Testing Tools? An Empirical Assessmentcited by 0
12 most cited of 15 All 15 (JSON)
- 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automation18 Aug 2026
- 2026ALIBI: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors via Adversarial Code Comments27 Jul 2026
- 2026Understanding Implicit Trust Errors in Core Carrier Networks through Multi-Agent Flaw Discovery and Analysis11 Jul 2026
- 2026RustMizan: A Compilable, Contamination-Aware Benchmarking Framework for Rust Vulnerabilities6 Jul 2026
- 2026Can Open-Source LLM Agents Replace Static Application Security Testing Tools? An Empirical Assessment10 Jun 2026
- 2026Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implications3 Jun 2026
- 2026Agora: Toward Autonomous Bug Detection in Production-Level Consensus Protocols with LLM Agents28 May 2026
- 2026FuzzingBrain V2: A Multi-Agent LLM System for Automated Vulnerability Discovery and Reproduction20 May 2026
- 2026Refusal Evaluation in Coding LLMs and Code Agents: A Systematic Review of Thirteen Malicious-Code Prompt Corpora (2023-2025)19 May 2026
- 2026Red-MIRROR: Agentic LLM-based Autonomous Penetration Testing with Reflective Verification and Knowledge-augmented Interaction28 Mar 2026
- 2026CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarking8 Feb 2026
- 2026A Dual-Loop Agent Framework for Automated Vulnerability Reproduction5 Feb 2026
12 newest of 15 All 15 (JSON)
- 2025VulnLLM-R: Specialized Reasoning LLM with Agent Scaffold for Vulnerability Detectioncited by 11
- 2026DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detectioncited by 3
- 2026Three Heads Are Better Than One: A Multi-perspective Reasoning Framework for Enhanced Vulnerability Detectioncited by 1
- 2026CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detectioncited by 0
- 2026CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detection4 Aug 2026
- 2026DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection15 Jul 2026
- 2026Three Heads Are Better Than One: A Multi-perspective Reasoning Framework for Enhanced Vulnerability Detection18 May 2026
- 2025VulnLLM-R: Specialized Reasoning LLM with Agent Scaffold for Vulnerability Detection8 Dec 2025
- 2025Large Language Models for Security Operations Centers: A Comprehensive Surveycited by 9
- 2025A Systematic Literature Review on Detecting Software Vulnerabilities with Large Language Modelscited by 8
- 2026Reducing False Positives in Static Bug Detection with LLMs: An Empirical Study in Industrycited by 5
- 2026Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Reviewcited by 3
- 2025Data and Context Matter: Towards Generalizing AI-based Software Vulnerability Detectioncited by 2
- 2025Securing the AI Supply Chain: What Can We Learn From Developer-Reported Security Issues and Solutions of AI Projects?cited by 1
- 2026EntailLLM: Verifying LLM-Generated Vulnerability Discovery Paths with Domain Knowledge via Logic Programmingcited by 1
- 2025On the Effectiveness of Instruction-Tuning Local LLMs for Identifying Software Vulnerabilitiescited by 0
- 2025Assessing the Software Security Comprehension of Large Language Modelscited by 0
- 2026RiskBridge: Turning CVEs into Business-Aligned Patch Prioritiescited by 0
- 2026Seclens: Role-specific Evaluation of LLM's for security vulnerablity detectioncited by 0
- 2026Improving MPI Error Detection and Repair with Large Language Models and Bug Referencescited by 0
12 most cited of 22 All 22 (JSON)
- 2026Automated Vulnerability Injection in Smart Contracts Using Large Language Models2 Sep 2026
- 2026FuzzingBrain-Bench V1: Evaluating Open-Ended Bug Discovery by LLMs25 Aug 2026
- 2026Large Language Models at the Intersection of Software Engineering and Software Security:An Evidence-Centered Structured Survey and Research Agenda21 Aug 2026
- 2026Cross-Corpus Evaluation of Generalizable Vulnerability Detection in IoT Firmware11 Aug 2026
- 2026EntailLLM: Verifying LLM-Generated Vulnerability Discovery Paths with Domain Knowledge via Logic Programming3 Aug 2026
- 2026Evaluating Large Language Models for Symbolic Security Protocol Analysis22 Jul 2026
- 2026From Adoption to Deployment: A Qualitative Study on AI Integration in Software Development Practice18 Jul 2026
- 2026Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection29 Jun 2026
- 2026Willing but Unable: Separating Refusal from Capability in Code LLMs via Abliteration3 Jun 2026
- 2026One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent7 May 2026
- 2026Learning Generalizable Multimodal Representations for Software Vulnerability Detection28 Apr 2026
- 2026Improving MPI Error Detection and Repair with Large Language Models and Bug References2 Apr 2026
12 newest of 22 All 22 (JSON)