# CyberRAG: An Agentic RAG cyber attack classification and reporting tool

arXiv: https://arxiv.org/abs/2507.02424 · HTML: https://mlsanity.com/paper/2507.02424/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 27 of its 78 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)
- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Pretrained Transformers Improve Out-of-Distribution Robustness](https://mlsanity.com/paper/2004.06100/) (2020)
- [Modular Multitask Reinforcement Learning with Policy Sketches](https://mlsanity.com/paper/1611.01796/) (2016)
- [AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways](https://mlsanity.com/paper/2406.02630/) (2024)
- [AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks](https://mlsanity.com/paper/2403.01038/) (2024)
- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [Large Language Models in Cybersecurity: State-of-the-Art](https://mlsanity.com/paper/2402.00891/) (2024)
- [SecBench: A Comprehensive Multi-Dimensional Benchmarking Dataset for LLMs in Cybersecurity](https://mlsanity.com/paper/2412.20787/) (2024)
- [Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks](https://mlsanity.com/paper/2410.20911/) (2024)
- [A RAG-Based Question-Answering Solution for Cyber-Attack Investigation and Attribution](https://mlsanity.com/paper/2408.06272/) (2024)
- [LLM Cyber Evaluations Don't Capture Real-World Risk](https://mlsanity.com/paper/2502.00072/) (2025)
- [TechniqueRAG: Retrieval Augmented Generation for Adversarial Technique Annotation in Cyber Threat Intelligence Text](https://mlsanity.com/paper/2505.11988/) (2025)
- [AURA: A Multi-Agent Intelligence Framework for Knowledge-Enhanced Cyber Threat Attribution](https://mlsanity.com/paper/2506.10175/) (2025)
- [Analysis of SQL Injection Detection Techniques](https://mlsanity.com/paper/1605.02796/) (2016)

## Built on by (most cited first, 11 of 11)

- [The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise](https://mlsanity.com/paper/2507.06850/) (2025)
- [Retrieval-Augmented LLMs for Security Incident Analysis](https://mlsanity.com/paper/2603.18196/) (2026)
- [AutoBnB-RAG: Enhancing Multi-Agent Incident Response with Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.13118/) (2025)
- [DiagChain: A Diagnostic Benchmark for Evaluating LLM Agents on Evidence-Grounded Attack Chain Reconstruction](https://mlsanity.com/paper/2608.03591/) (2026)
- [CyberAId: AI-Driven Cybersecurity for Financial Service Providers](https://mlsanity.com/paper/2605.01892/) (2026)
- [A Multi-Layer Cloud-IDS Pipeline with LLM and Adaptive Q-Learning Calibration](https://mlsanity.com/paper/2605.15889/) (2026)
- [From Detection to Response: A Deep Learning and Retrieval-Augmented Generation Framework for Network Intrusion Mitigation](https://mlsanity.com/paper/2605.17960/) (2026)
- [An Organization-Scoped LLM Agent Runtime Architecture for Regulated Cybersecurity Operations](https://mlsanity.com/paper/2605.30604/) (2026)
- [MIRROR: Novelty-Constrained Memory-Guided MCTS Red-Teaming for Agentic RAG](https://mlsanity.com/paper/2606.26793/) (2026)
- [Defending Retrieval-Augmented Intrusion Detection Against Knowledge Poisoning and Prompt Injection](https://mlsanity.com/paper/2608.08100/) (2026)
- [Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making](https://mlsanity.com/paper/2608.20966/) (2026)

## Abstract (arXiv)

Intrusion Detection and Prevention Systems (IDS/IPS) in large enterprises can generate hundreds of thousands of alerts per hour, overwhelming analysts with logs requiring rapidly evolving expertise. Conventional machine-learning detectors reduce alert volume but still yield many false positives, while standard Retrieval-Augmented Generation (RAG) pipelines often retrieve irrelevant context and fail to justify predictions. We present CyberRAG, a modular agent-based RAG framework that delivers real-time classification, explanation, and structured reporting for cyber-attacks. A central LLM agent orchestrates: (i) fine-tuned classifiers specialized by attack family; (ii) tool adapters for enrichment and alerting; and (iii) an iterative retrieval-and-reason loop that queries a domain-specific knowledge base until evidence is relevant and self-consistent. Unlike traditional RAG, CyberRAG adopts an agentic design that enables dynamic control flow and adaptive reasoning. This architecture autonomously refines threat labels and natural-language justifications, reducing false positives and enhancing interpretability. It is also extensible: new attack types can be supported by adding classifiers without retraining the core agent. CyberRAG was evaluated on SQL Injection, XSS, and SSTI, achieving over 94% accuracy per class and a final classification accuracy of 94.92% through semantic orchestration. Generated explanations reached 0.94 in BERTScore and 4.9/5 in GPT-4-based expert evaluation, with robustness preserved against adversarial and unseen payloads. These results show that agentic, specialist-oriented RAG can combine high detection accuracy with trustworthy, SOC-ready prose, offering a flexible path toward partially automated cyber-defense workflows.