mlsanityData to 7 Oct 2026

LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights

arXiv 2502.07049 ↗First posted 10 Feb 2025cs.CR cs.AIcited by 44

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are emerging as transformative tools for software vulnerability detection, addressing critical challenges in the security domain. Traditional methods, such as static and dynamic analysis, often falter due to inefficiencies, high false positive rates, and the growing complexity of modern software systems. By leveraging their ability to analyze code structures, identify patterns, and generate repair suggestions, LLMs, exemplified by models like GPT, BERT, and CodeBERT, present a novel and scalable approach to mitigating vulnerabilities. This paper provides a detailed survey of LLMs in vulnerability detection. It examines key aspects, including model architectures, application methods, target languages, fine-tuning strategies, datasets, and evaluation metrics. We also analyze the scope of current research problems, highlighting the strengths and weaknesses of existing approaches. Further, we address challenges such as cross-language vulnerability detection, multimodal data integration, and repository-level analysis. Based on these findings, we propose solutions for issues like dataset scalability, model interpretability, and applications in low-resource scenarios. Our contributions are threefold: (1) a systematic review of how LLMs are applied in vulnerability detection; (2) an analysis of shared patterns and differences across studies, with a unified framework for understanding the field; and (3) a summary of key challenges and future research directions. This work provides valuable insights for advancing LLM-based vulnerability detection. We also maintain and regularly update latest selected paper on https://github.com/OwenSanzas/LLM-For-Vulnerability-Detection

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 73 of its 135 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

73 of 135 references.

References · 73

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  7. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  8. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  9. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  10. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  11. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  12. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
38 more
  1. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  2. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  3. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  4. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  5. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 355
  6. 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 266
  7. 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 149
  8. 2024Vulnerability Detection with Code Language Models: How Far Are We?cited by 133
  9. 2023AugGPT: Leveraging ChatGPT for Text Data Augmentationcited by 128
  10. 2024LLM Agents can Autonomously Exploit One-day Vulnerabilitiescited by 125
  11. 2024IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilitiescited by 100
  12. 2023DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detectioncited by 98
  13. 2023LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarkscited by 94
  14. 2018SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilitiescited by 91
  15. 2021CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Softwarecited by 89
  16. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  17. 2023Large Language Models for Test-Free Fault Localizationcited by 67
  18. 2024Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAGcited by 67
  19. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  20. 2023Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilitiescited by 58
  21. 2022NatGen: Generative pre-training by "Naturalizing" source codecited by 53
  22. 2022VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detectioncited by 51
  23. 2023How Effective Are Neural Networks for Fixing Security Vulnerabilitiescited by 50
  24. 2021D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysiscited by 48
  25. 2023Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectivescited by 48
  26. 2023ChatGPT for Vulnerability Detection, Classification, and Repair: How Far Are We?cited by 44
  27. 2024Combining Fine-Tuning and LLM-based Agents for Intuitive Smart Contract Auditing with Justificationscited by 41
  28. 2024Finetuning Large Language Models for Vulnerability Detectioncited by 39
  29. 2020VulDeeLocator: A Deep Learning-based Fine-grained Vulnerability Detectorcited by 37
  30. 2024Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuningcited by 30
  31. 2019A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Softwarecited by 28
  32. 2023Prompt Fuzzing for Fuzz Driver Generationcited by 27
  33. 2024How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queriescited by 26
  34. 2023Nuances are the Key: Unlocking ChatGPT to Find Failure-Inducing Tests with Differential Promptingcited by 23
  35. 2024Multitask-based Evaluation of Open-Source LLM on Software Vulnerabilitycited by 23
  36. 2023How Effective Are They? Exploring Large Language Model Based Fuzz Driver Generationcited by 21
  37. 2020How Effective are Smart Contract Analysis Tools? Evaluating Smart Contract Static Analysis Tools Using Bug Injectioncited by 20
  38. 2024Multi-role Consensus through LLMs Discussions for Vulnerability Detectioncited by 19

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 33.

44 papers.

Sort:

By area:

  1. 2025VulnLLM-R: Specialized Reasoning LLM with Agent Scaffold for Vulnerability Detectioncited by 11
  2. 2025Large Language Models for Security Operations Centers: A Comprehensive Surveycited by 9
  3. 2025A Systematic Literature Review on Detecting Software Vulnerabilities with Large Language Modelscited by 8
  4. 2026MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolutioncited by 6
  5. 2025How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-benchcited by 5
  6. 2026Reducing False Positives in Static Bug Detection with LLMs: An Empirical Study in Industrycited by 5
  7. 2025AutoPatch: Multi-Agent Framework for Patching Real-World CVE Vulnerabilitiescited by 4
  8. 2025Agentic Discovery and Validation of Android App Vulnerabilitiescited by 4
  9. 2026CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarkingcited by 3
  10. 2026Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Reviewcited by 3
  11. 2026FuzzingBrain V2: A Multi-Agent LLM System for Automated Vulnerability Discovery and Reproductioncited by 3
  12. 2026DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detectioncited by 3
32 more
  1. 2025Data and Context Matter: Towards Generalizing AI-based Software Vulnerability Detectioncited by 2
  2. 2025Unified Threat Detection and Mitigation Framework (UTDMF): Combating Prompt Injection, Deception, and Bias in Enterprise-Scale Transformerscited by 1
  3. 2025Securing the AI Supply Chain: What Can We Learn From Developer-Reported Security Issues and Solutions of AI Projects?cited by 1
  4. 2026A Dual-Loop Agent Framework for Automated Vulnerability Reproductioncited by 1
  5. 2026Red-MIRROR: Agentic LLM-based Autonomous Penetration Testing with Reflective Verification and Knowledge-augmented Interactioncited by 1
  6. 2026Three Heads Are Better Than One: A Multi-perspective Reasoning Framework for Enhanced Vulnerability Detectioncited by 1
  7. 2026Refusal Evaluation in Coding LLMs and Code Agents: A Systematic Review of Thirteen Malicious-Code Prompt Corpora (2023-2025)cited by 1
  8. 2026Agora: Toward Autonomous Bug Detection in Production-Level Consensus Protocols with LLM Agentscited by 1
  9. 2026Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implicationscited by 1
  10. 2026RustMizan: A Compilable, Contamination-Aware Benchmarking Framework for Rust Vulnerabilitiescited by 1
  11. 2026EntailLLM: Verifying LLM-Generated Vulnerability Discovery Paths with Domain Knowledge via Logic Programmingcited by 1
  12. 2025On the Effectiveness of Instruction-Tuning Local LLMs for Identifying Software Vulnerabilitiescited by 0
  13. 2025Assessing the Software Security Comprehension of Large Language Modelscited by 0
  14. 2026RiskBridge: Turning CVEs into Business-Aligned Patch Prioritiescited by 0
  15. 2026Seclens: Role-specific Evaluation of LLM's for security vulnerablity detectioncited by 0
  16. 2026Improving MPI Error Detection and Repair with Large Language Models and Bug Referencescited by 0
  17. 2026Learning Generalizable Multimodal Representations for Software Vulnerability Detectioncited by 0
  18. 2026One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intentcited by 0
  19. 2026Willing but Unable: Separating Refusal from Capability in Code LLMs via Abliterationcited by 0
  20. 2026Can Open-Source LLM Agents Replace Static Application Security Testing Tools? An Empirical Assessmentcited by 0
  21. 2026Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detectioncited by 0
  22. 2026Understanding Implicit Trust Errors in Core Carrier Networks through Multi-Agent Flaw Discovery and Analysiscited by 0
  23. 2026From Adoption to Deployment: A Qualitative Study on AI Integration in Software Development Practicecited by 0
  24. 2026Evaluating Large Language Models for Symbolic Security Protocol Analysiscited by 0
  25. 2026ALIBI: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors via Adversarial Code Commentscited by 0
  26. 2026CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detectioncited by 0
  27. 2026Cross-Corpus Evaluation of Generalizable Vulnerability Detection in IoT Firmwarecited by 0
  28. 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automationcited by 0
  29. 2026Large Language Models at the Intersection of Software Engineering and Software Security:An Evidence-Centered Structured Survey and Research Agendacited by 0
  30. 2026FuzzingBrain-Bench V1: Evaluating Open-Ended Bug Discovery by LLMscited by 0
  31. 2026Automated Vulnerability Injection in Smart Contracts Using Large Language Modelscited by 0
  32. 2026Prompt Injection Detection for Email Agents Through Attack Chain Modelingcited by 0