mlsanityData to 14 Oct 2026

One Bug, Hundreds Behind: LLMs for Large-Scale Bug Discovery

arXiv 2510.14036 ↗First posted 15 Oct 2025cs.SE cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Fixing bugs in large programs is a challenging task that demands substantial time and effort. Once a bug is found, it is reported to the project maintainers, who work with the reporter to fix it and eventually close the issue. However, across the program, there are often similar code segments, which may also contain the bug, but were missed during discovery. Finding and fixing each recurring bug instance individually is labor intensive. Even more concerning, bug reports can inadvertently widen the attack surface as they provide attackers with an exploitable pattern that may be unresolved in other parts of the program. In this paper, we explore these Recurring Pattern Bugs (RPBs) that appear repeatedly across various code segments of a program or even in different programs, stemming from a same root cause, but are unresolved. Our investigation reveals that RPBs are widespread and can significantly compromise the security of software programs. This paper introduces BugStone, a program analysis system empowered by LLVM and a Large Language Model (LLM). The key observation is that many RPBs have one patched instance, which can be leveraged to identify a consistent error pattern, such as a specific API misuse. By examining the entire program for this pattern, it is possible to identify similar sections of code that may be vulnerable. Starting with 135 unique RPBs, BugStone identified more than 22K new potential issues in the Linux kernel. Manual analysis of 400 of these findings confirmed that 246 were valid. We also created a dataset from over 1.9K security bugs reported by 23 recent top-tier conference works. We manually annotate the dataset, identify 80 recurring patterns and 850 corresponding fixes. Even with a cost-efficient model choice, BugStone achieved 92.2% precision and 79.1% pairwise accuracy on the dataset.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 18 of its 86 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

18 of 86 references.

References · 18

  1. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  2. 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 272
  3. 2020Deep Learning based Vulnerability Detection: Are We There Yet?cited by 144
  4. 2024Vulnerability Detection with Code Language Models: How Far Are We?cited by 139
  5. 2024IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilitiescited by 103
  6. 2023LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarkscited by 97
  7. 2024Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAGcited by 68
  8. 2023Large Language Models for Test-Free Fault Localizationcited by 67
  9. 2024Combining Fine-Tuning and LLM-based Agents for Intuitive Smart Contract Auditing with Justificationscited by 41
  10. 2024Finetuning Large Language Models for Vulnerability Detectioncited by 40
  11. 2024Multitask-based Evaluation of Open-Source LLM on Software Vulnerabilitycited by 23
  12. 2023How Can ChatGPT Support Human Security Testers to Help Mitigate Supply Chain Attacks?cited by 20
6 more
  1. 2023Exploring ChatGPT's Capabilities on Vulnerability Managementcited by 19
  2. 2024Assessing LLMs in Malicious Code Deobfuscation of Real-world Malware Campaignscited by 19
  3. 2023SecureFalcon: Are We There Yet in Automated Software Vulnerability Detection with LLMs?cited by 13
  4. 2025Rethinking the Evaluation of Secure Code Generationcited by 10
  5. 2024Assessing the Effectiveness of LLMs in Android Application Vulnerability Analysiscited by 6
  6. 2023Gitor: Scalable Code Clone Detection by Building Global Sample Graphcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026GBQA: A Game Benchmark for Evaluating LLMs as Quality Assurance Engineerscited by 5
  2. 2026AnyPoC: Universal Proof-of-Concept Test Generation for Scalable LLM-Based Bug Detectioncited by 4
  3. 2026Comprehensive AI governance requires addressing non-model gainscited by 1
  4. 2026Revelio: Cost-Efficient Agentic Memory Safety Vulnerability Detection For Repository-Scale Codebasescited by 1
  5. 2026PrimSynth: An Agentic Approach to Discover, Validate, and Synthesize Exploit Primitives for Linux Kernel Vulnerabilitiescited by 0
  6. 2026The History Is the Detector: Executing CVE Patch History, End-to-Endcited by 0