mlsanityData to 14 Oct 2026

Investigating Large Language Models for Code Vulnerability Detection: An Experimental Study

arXiv 2412.18260 ↗First posted 24 Dec 2024cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Code vulnerability detection (CVD) is essential for addressing and preventing system security issues, playing a crucial role in ensuring software security. Previous learning-based vulnerability detection methods rely on either fine-tuning medium-size sequence models or training smaller neural networks from scratch. Recent advancements in large pre-trained language models (LLMs) have showcased remarkable capabilities in various code intelligence tasks including code understanding and generation. However, the effectiveness of LLMs in detecting code vulnerabilities is largely under-explored. This work aims to investigate the gap by fine-tuning LLMs for the CVD task, involving four widely-used open-source LLMs. We also implement other five previous graph-based or medium-size sequence models for comparison. Experiments are conducted on five commonly-used CVD datasets, including both the part of short samples and long samples. In addition, we conduct quantitative experiments to investigate the class imbalance issue and the model's performance on samples of different lengths, which are rarely studied in previous works. To better facilitate communities, we open-source all codes and resources of this study in https://github.com/SakiRinn/LLM4CVD and https://huggingface.co/datasets/xuefen/VulResource.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 49 of its 74 references that are in the map. First posted on arXiv 24 Dec 2024.

Builds on

49 of 74 references.

References · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2024The Llama 3 Herd of Modelscited by 14,715
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  11. 2023Mistral 7Bcited by 3,164
  12. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
37 more
  1. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  3. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  4. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  5. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
  6. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  7. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  8. 2023The Falcon Series of Open Language Modelscited by 535
  9. 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 420
  10. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 361
  11. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 300
  12. 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 272
  13. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  14. 2020Deep Learning based Vulnerability Detection: Are We There Yet?cited by 144
  15. 2017L2 Regularization versus Batch and Weight Normalizationcited by 124
  16. 2023DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detectioncited by 102
  17. 2018SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilitiescited by 92
  18. 2023On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilotcited by 64
  19. 2023Data Quality for Software Vulnerability Datasetscited by 51
  20. 2021D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysiscited by 49
  21. 2022An Empirical Study of Deep Learning Models for Vulnerability Detectioncited by 46
  22. 2024Large Language Model for Vulnerability Detection: Emerging Results and Future Directionscited by 45
  23. 2023BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Modelscited by 42
  24. 2021ReGVD: Revisiting Graph Neural Networks for Vulnerability Detectioncited by 32
  25. 2023Fair Text-to-Image Diffusion via Fair Mappingcited by 31
  26. 2024Chain-of-Thought Prompting of Large Language Models for Discovering and Fixing Software Vulnerabilitiescited by 31
  27. 2024Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuningcited by 30
  28. 2023How Far Have We Gone in Vulnerability Detection Using Large Language Modelscited by 27
  29. 2024MegaVul: A C/C++ Vulnerability Dataset with Comprehensive Code Representationcited by 27
  30. 2022Towards Federated Learning against Noisy Labels via Local Self-Regularizationcited by 26
  31. 2023Deep Learning for Code Intelligence: Survey, Benchmark and Toolkitcited by 22
  32. 2023Pre-trained Model-based Automated Software Vulnerability Repair: How Far are We?cited by 16
  33. 2024Tackling Noisy Clients in Federated Learning with End-to-end Label Correctioncited by 12
  34. 2023The Devil is in the Tails: How Long-Tailed Code Distributions Impact Large Language Modelscited by 8
  35. 2021CGEMs: A Metric Model for Automatic Code Generation using GPT-3cited by 6
  36. 2024Data Quality Issues in Vulnerability Detection Datasetscited by 5
  37. 2024FedLF: Adaptive Logit Adjustment and Feature Optimization in Federated Long-Tailed Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 5.

12 papers.

  1. 2025SafeGenBench: A Benchmark Framework for Security Vulnerability Detection in LLM-Generated Codecited by 19
  2. 2025Vulnerability Detection: From Formal Verification to Large Language Models and Hybrid Approaches: A Comprehensive Overviewcited by 7
  3. 2025R2Vul: Learning to Reason about Software Vulnerabilities with Reinforcement Learning and Structured Reasoning Distillationcited by 7
  4. 2025Revisiting Pre-trained Language Models for Vulnerability Detectioncited by 3
  5. 2025Llama-based source code vulnerability detection: Prompt engineering vs Fine tuningcited by 2
  6. 2025ZoFia: Zero-Shot Fake News Detection with Entity-Guided Retrieval and Multi-LLM Interactioncited by 1
  7. 2025LLaVul: A Multimodal LLM for Interpretable Vulnerability Reasoning about Source Codecited by 0
  8. 2026Beyond Function-Level Analysis: Context-Aware Reasoning for Inter-Procedural Vulnerability Detectioncited by 0
  9. 2026VulReaD: Knowledge-Graph-guided Software Vulnerability Reasoning and Detectioncited by 0
  10. 2026Multi-View Decompilation for LLM-Based Malware Classificationcited by 0
  11. 2026Abliteration Mitigation via Refusal Aliasescited by 0
  12. 2026Diagnosing Generalization Boundaries in AI-Based Vulnerability Detection via Calibrated Embedding Distancescited by 0