QASecClaw: A Multi-Agent LLM Approach for False Positive Reduction in Static Application Security Testing
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Static Application Security Testing tools help developers find security vulnerabilities before release, but they often produce many false positives. This increases manual review effort, reduces developer trust, and may cause real vulnerabilities to be ignored among noisy reports. We present QASecClaw, a multi agent approach that combines conventional Static Application Security Testing with coding specialized Large Language Model based contextual code review. A SAST engine first reports candidate vulnerabilities, and a Large Language Model based SAST Filter Agent then reviews each finding with source code context to decide whether it is likely to be a true positive or a false positive. QASecClaw is coordinated by a Mission Orchestrator and includes specialized agents for test planning, security validation, evidence correlation, filtering, and reporting. We evaluate QASecClaw on OWASP Benchmark v1.2, which contains 2,740 Java test cases across 11 Common Weakness Enumeration categories with ground truth labels. QASecClaw achieves an F1 score of 90.93 percent, compared with 78.39 percent for standalone Semgrep. The improvement is mainly driven by an 88.6 percent reduction in false positives, from 560 to 64, with only a 3.1 percent reduction in recall. These results show that Large Language Model augmented multi agent verification can make Static Application Security Testing output more accurate, useful, and trustworthy.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 16 of its 35 references that are in the map. First posted on arXiv 3 May 2026.
Builds on
16 of 35 references.
References · 16
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2024Qwen2 Technical Reportcited by 2,120
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
- 2023ChatDev: Communicative Agents for Software Developmentcited by 919
- 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 272
- 2020Deep Learning based Vulnerability Detection: Are We There Yet?cited by 144
- 2022Practical Program Repair in the Era of Large Pre-trained Language Modelscited by 133
- 2021Examining Zero-Shot Vulnerability Repair with Large Language Modelscited by 105
- 2023Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilitiescited by 59
4 more
- 2024LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoningcited by 59
- 2022An Empirical Study of Deep Learning Models for Vulnerability Detectioncited by 46
- 2023Evaluation of ChatGPT Model for Vulnerability Detectioncited by 33
- 2023The Hitchhiker's Guide to Program Analysis: A Journey with Large Language Modelscited by 18
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025QLCoder: A Query Synthesizer For Static Analysis of Security Vulnerabilities
- 2026Argus: Reorchestrating Static Analysis via a Multi-Agent Ensemble for Full-Chain Security Vulnerability Detection
- 2026When Passing Tests Hides Vulnerabilities: An Empirical Study of Silent Failures in Agentic Systems
- 2024AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testing
- 2025CodeCureAgent: Automatic Classification and Repair of Static Analysis Warnings
- 2026An Empirical Analysis of CodeQL False Positives and Query Refinements for Java Vulnerabilities ↗
- 2026Sifting the Noise: A Comparative Study of LLM Agents in Vulnerability False Positive Filtering
- 2025Let the Trial Begin: A Mock-Court Approach to Vulnerability Detection using LLM-Based Agents