mlsanityData to 14 Oct 2026

Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG

arXiv 2406.11147 ↗First posted 17 Jun 2024cs.SE cs.AIcited by 68

Not on the map. Why?

Cited by 24 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Although LLMs have shown promising potential in vulnerability detection, this study reveals their limitations in distinguishing between vulnerable and similar-but-benign patched code (only 0.06 - 0.14 accuracy). It shows that LLMs struggle to capture the root causes of vulnerabilities during vulnerability detection. To address this challenge, we propose enhancing LLMs with multi-dimensional vulnerability knowledge distilled from historical vulnerabilities and fixes. We design a novel knowledge-level Retrieval-Augmented Generation framework Vul-RAG, which improves LLMs with an accuracy increase of 16% - 24% in identifying vulnerable and patched code. Additionally, vulnerability knowledge generated by Vul-RAG can further (1) serve as high-quality explanations to improve manual detection accuracy (from 60% to 77%), and (2) detect 10 previously-unknown bugs in the recent Linux kernel release with 6 assigned CVEs.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Builds on 40 of its 80 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

40 of 80 references.

References · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  8. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  9. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
  10. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  11. 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 272
  12. 2020Deep Learning based Vulnerability Detection: Are We There Yet?cited by 144
28 more
  1. 2024Vulnerability Detection with Code Language Models: How Far Are We?cited by 139
  2. 2023Structured Chain-of-Thought Prompting for Code Generationcited by 138
  3. 2022Few-shot training LLMs for project-specific code-summarizationcited by 129
  4. 2022ReACC: A Retrieval-Augmented Code Completion Frameworkcited by 119
  5. 2024IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilitiescited by 103
  6. 2023DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detectioncited by 102
  7. 2023LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarkscited by 97
  8. 2021CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Softwarecited by 91
  9. 2023Large Language Models for Test-Free Fault Localizationcited by 67
  10. 2023Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilitiescited by 59
  11. 2024LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoningcited by 59
  12. 2022VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detectioncited by 51
  13. 2023Data Quality for Software Vulnerability Datasetscited by 51
  14. 2023Prompt-Enhanced Software Vulnerability Detection Using ChatGPTcited by 51
  15. 2024Large Language Model for Vulnerability Detection: Emerging Results and Future Directionscited by 45
  16. 2023RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repaircited by 44
  17. 2024Large Language Model for Vulnerability Detection and Repair: Literature Review and the Road Aheadcited by 42
  18. 2024Finetuning Large Language Models for Vulnerability Detectioncited by 40
  19. 2024Chain-of-Thought Prompting of Large Language Models for Discovering and Fixing Software Vulnerabilitiescited by 31
  20. 2023Uncovering the Limits of Machine Learning for Automatic Vulnerability Detectioncited by 30
  21. 2022RACE: Retrieval-Augmented Commit Message Generationcited by 27
  22. 2023How Far Have We Gone in Vulnerability Detection Using Large Language Modelscited by 27
  23. 2022Dataflow Analysis-Inspired Deep Learning for Efficient Vulnerability Detectioncited by 19
  24. 2023The Hitchhiker's Guide to Program Analysis: A Journey with Large Language Modelscited by 18
  25. 2024LLbezpeky: Leveraging Large Language Models for Vulnerability Detectioncited by 13
  26. 2024Toward Improved Deep Learning-based Vulnerability Detectioncited by 9
  27. 2023Exploring the Limits of ChatGPT in Software Security Applicationscited by 6
  28. 2023Exploring Large Language Models in Resolving Environment-Related Crash Bugs: Localizing and Repairingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 28; 2026: 34.

68 papers.

Sort:

By area:

  1. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 178
  2. 2024Agents in Software Engineering: Survey, Landscape, and Visioncited by 60
  3. 2025LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insightscited by 47
  4. 2025Benchmarking LLMs and LLM-based Agents in Practical Vulnerability Detection for Code Repositoriescited by 31
  5. 2025KNighter: Transforming Static Analysis with LLM-Synthesized Checkerscited by 24
  6. 2025CoRe: Benchmarking LLMs Code Reasoning Capabilities through Static Analysis Taskscited by 24
  7. 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
  8. 2025Let the Trial Begin: A Mock-Court Approach to Vulnerability Detection using LLM-Based Agentscited by 20
  9. 2025A Framework for Evaluating Emerging Cyberattack Capabilities of AIcited by 16
  10. 2025CRAKEN: Cybersecurity LLM Agent with Knowledge-Based Executioncited by 14
  11. 2025VulnLLM-R: Specialized Reasoning LLM with Agent Scaffold for Vulnerability Detectioncited by 11
  12. 2025Large Language Models for Security Operations Centers: A Comprehensive Surveycited by 9
38 more
  1. 2024LProtector: An LLM-driven Vulnerability Detection Systemcited by 8
  2. 2024VulScribeR: Exploring RAG-based Vulnerability Augmentation with LLMscited by 7
  3. 2025R2Vul: Learning to Reason about Software Vulnerabilities with Reinforcement Learning and Structured Reasoning Distillationcited by 7
  4. 2024ProveRAG: Provenance-Driven Vulnerability Analysis with Automated Retrieval-Augmented LLMscited by 6
  5. 2025One Bug, Hundreds Behind: LLMs for Large-Scale Bug Discoverycited by 6
  6. 2025SecureReviewer: Enhancing Large Language Models for Secure Code Review through Secure-aware Fine-tuningcited by 5
  7. 2025A Systematic Study of Code Obfuscation Against LLM-based Vulnerability Detectioncited by 5
  8. 2026Reducing False Positives in Static Bug Detection with LLMs: An Empirical Study in Industrycited by 5
  9. 2025AutoPatch: Multi-Agent Framework for Patching Real-World CVE Vulnerabilitiescited by 4
  10. 2025Improving LLM Reasoning for Vulnerability Detection via Group Relative Policy Optimizationcited by 4
  11. 2025A Mixture of Linear Corrections Generates Secure Codecited by 4
  12. 2025VulInstruct: Teaching LLMs Root-Cause Reasoning for Vulnerability Detection via Security Specificationscited by 4
  13. 2026From SFT to RL: Demystifying the Post-Training Pipeline for LLM-based Vulnerability Detectioncited by 4
  14. 2026AnyPoC: Universal Proof-of-Concept Test Generation for Scalable LLM-Based Bug Detectioncited by 4
  15. 2025Detecting Malicious Source Code in PyPI Packages with LLMs: Does RAG Come in Handy?cited by 3
  16. 2025Trust Me, I Know This Function: Hijacking LLM Static Analysis using Biascited by 3
  17. 2026FuzzingBrain V2: A Multi-Agent LLM System for Automated Vulnerability Discovery and Reproductioncited by 3
  18. 2026DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detectioncited by 3
  19. 2025LLM Embedding-based Attribution (LEA): Quantifying Source Contributions to Generative Model's Response for Vulnerability Analysiscited by 2
  20. 2026Revisiting Vul-RAG: Reproducibility and Replicability of RAG-based Vulnerability Detection with Open-Weight Modelscited by 2
  21. 2025Automated detection of atomicity violations in large-scale systemscited by 1
  22. 2025BlueCodeAgent: A Blue Teaming Agent Enabled by Automated Red Teaming for CodeGen AIcited by 1
  23. 2026AEGIS: From Clues to Verdicts -- Graph-Guided Deep Vulnerability Reasoning via Dialectics and Meta-Auditingcited by 1
  24. 2026Knowdit: Agentic Smart Contract Vulnerability Detection with Auditing Knowledge Summarizationcited by 1
  25. 2026MemBoost: A Memory-Boosted Framework for Cost-Aware LLM Inferencecited by 1
  26. 2026Three Heads Are Better Than One: A Multi-perspective Reasoning Framework for Enhanced Vulnerability Detectioncited by 1
  27. 2026Comprehensive AI governance requires addressing non-model gainscited by 1
  28. 2026Revelio: Cost-Efficient Agentic Memory Safety Vulnerability Detection For Repository-Scale Codebasescited by 1
  29. 2026DeepDiscovery: A Location-Inference Framework for Task-Level Repository Understandingcited by 1
  30. 2026An Interactive Agent for Requirement-Driven Candidate Sourcingcited by 1
  31. 2026CodePoisonRAG: Knowledge Poisoning Attacks on Retrieval-Augmented Code Generationcited by 1
  32. 2024Direction for Detection: A Survey of Automated Vulnerability Detection and all of its Pain Pointscited by 0
  33. 2025iPanda: An LLM-based Agent for Automated Conformance Testing of Communication Protocolscited by 0
  34. 2025WALL: A Web Application for Automated Quality Assurance using Large Language Modelscited by 0
  35. 2025Orion: Fuzzing Workflow Automationcited by 0
  36. 2025On the Difficulty of Selecting Few-Shot Examples for Effective LLM-based Vulnerability Detectioncited by 0
  37. 2025ReVul-CoT: Towards Effective Software Vulnerability Assessment with Retrieval-Augmented Generation and Chain-of-Thought Promptingcited by 0
  38. 2026An Empirical Evaluation of LLM-Based Approaches for Code Vulnerability Detection: RAG, SFT, and Dual-Agent Systemscited by 0

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026VulTriage: Triple-Path Context Augmentation for LLM-Based Vulnerability Detection
  2. 2026VulWeaver: Weaving Broken Semantics for Grounded Vulnerability Detection
  3. 2025Reasoning with LLMs for Zero-Shot Vulnerability Detection
  4. 2026Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software
  5. 2024A Case Study of LLM for Automated Vulnerability Repair: Assessing Impact of Reasoning and Patch Validation Feedback
  6. 2026VulnGym: Benchmarking Coding Agents for Repository-Level Vulnerability Detection
  7. 2024Towards Explainable Vulnerability Detection with Large Language Models
  8. 2025The Hitchhiker's Guide to Program Analysis, Part II: Deep Thoughts by LLMs