# DefectHunter: A Novel LLM-Driven Boosted-Conformer-based Code Vulnerability Detection Mechanism

arXiv: https://arxiv.org/abs/2309.15324 · HTML: https://mlsanity.com/paper/2309.15324/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2023.

## Built on by (most cited first, 14 of 14)

- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights](https://mlsanity.com/paper/2502.07049/) (2025)
- [LLbezpeky: Leveraging Large Language Models for Vulnerability Detection](https://mlsanity.com/paper/2401.01269/) (2024)
- [A Contemporary Survey of Large Language Model Assisted Program Analysis](https://mlsanity.com/paper/2502.18474/) (2025)
- [Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions](https://mlsanity.com/paper/2502.14202/) (2025)
- [RealVul: Can We Detect Vulnerabilities in Web Applications with LLM?](https://mlsanity.com/paper/2410.07573/) (2024)
- [PatUntrack: Automated Generating Patch Examples for Issue Reports without Tracked Insecure Code](https://mlsanity.com/paper/2408.08619/) (2024)
- [Generative AI for Internet of Things Security: Challenges and Opportunities](https://mlsanity.com/paper/2502.08886/) (2025)
- [LLMpatronous: Harnessing the Power of LLMs For Vulnerability Detection](https://mlsanity.com/paper/2504.18423/) (2025)
- [What Do They Fix? LLM-Aided Categorization of Security Patches for Critical Memory Bugs](https://mlsanity.com/paper/2509.22796/) (2025)
- [Direction for Detection: A Survey of Automated Vulnerability Detection and all of its Pain Points](https://mlsanity.com/paper/2412.11194/) (2024)
- [Automating the Detection of Code Vulnerabilities by Analyzing GitHub Issues](https://mlsanity.com/paper/2501.05258/) (2025)
- [JupOtter: Cell-Level Bug Detection in Jupyter Notebooks](https://mlsanity.com/paper/2606.23877/) (2026)

## Abstract (arXiv)

One of the most pressing threats to computing systems is software vulnerabilities, which can compromise both hardware and software components. Existing methods for vulnerability detection remain suboptimal. Traditional techniques are both time-consuming and labor-intensive, while machine-learning-based approaches often underperform when applied to complex datasets, due to their inability to capture high-dimensional relationships. Previous deep-learning strategies also fall short in capturing sufficient feature information. Although self-attention mechanisms can process information over long distances, they fail to capture structural information. In this paper, we introduce DefectHunter, an innovative model for vulnerability identification that employs the Conformer mechanism. This mechanism fuses self-attention with convolutional networks to capture both local, position-wise features and global, content-based interactions. Furthermore, we optimize the self-attention mechanisms to mitigate the issue of excessive attention heads introducing extraneous noise by adjusting the denominator. We evaluated DefectHunter against ten baseline methods using six industrial and two highly complex datasets. On the QEMU dataset, DefectHunter exhibited a 20.62% improvement in accuracy over Pongo-70B, and for the CWE-754 dataset, its accuracy was 14.64% higher. To investigate how DefectHunter comprehends vulnerabilities, we conducted a case study, which revealed that our model effectively understands the mechanisms underlying vulnerabilities.