mlsanityData to 7 Oct 2026

Retrieval Augmented Generation Based LLM Evaluation For Protocol State Machine Inference With Chain-of-Thought Reasoning

arXiv 2502.15727 ↗First posted 30 Jan 2025cs.NI cs.AI cs.CRcited by 4

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 2 more
Abstract (arXiv)

This paper presents a novel approach to evaluate the efficiency of a RAG-based agentic Large Language Model (LLM) architecture for network packet seed generation and enrichment. Enhanced by chain-of-thought (COT) prompting techniques, the proposed approach focuses on the improvement of the seeds' structural quality in order to guide protocol fuzzing frameworks through a wide exploration of the protocol state space. Our method leverages RAG and text embeddings to dynamically reference to the Request For Comments (RFC) documents knowledge base for answering queries regarding the protocol's Finite State Machine (FSM), then iteratively reasons through the retrieved knowledge, for output refinement and proper seed placement. We then evaluate the response structure quality of the agent's output, based on metrics as BLEU, ROUGE, and Word Error Rate (WER) by comparing the generated packets against the ground-truth packets. Our experiments demonstrate significant improvements of up to 18.19%, 14.81%, and 23.45% in BLEU, ROUGE, and WER, respectively, over baseline models. These results confirm the potential of such approach, improving LLM-based protocol fuzzing frameworks for the identification of hidden vulnerabilities.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 10 of its 23 references that are in the map. First posted on arXiv 30 Jan 2025.

Builds on

10 of 23 references.

References · 10

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2023Augmenting Greybox Fuzzing with Generative AIcited by 18
  4. 2024Harnessing Large Language Models for Seed Generation in Greybox Fuzzingcited by 8
  5. 2024LLM-DaaS: LLM-driven Drone-as-a-Service Operations from Text User Requestscited by 7
  6. 2024LLM-SEM: A Sentiment-Based Student Engagement Metric Using LLMS for E-Learning Platformscited by 6
  7. 2024RIRO: Reshaping Inputs, Refining Outputs Unlocking the Potential of Large Language Models in Data-Scarce Contextscited by 6
  8. 2024LMRPA: Large Language Model-Driven Efficient Robotic Process Automation for OCRcited by 6
  9. 2024LMV-RPA: Large Model Voting-based Robotic Process Automationcited by 5
  10. 2024Unleashing the Power of LLM to Infer State Machine from the Protocol Implementationcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025RAGXplain: From Explainable Evaluation to Actionable Guidance of RAG Pipelinescited by 4
  2. 2025An Agentic Flow for Finite State Machine Extraction using Prompt Chainingcited by 0
  3. 2025MultiFuzz: A Dense Retrieval-based Multi-Agent System for Network Protocol Fuzzingcited by 0
  4. 2025An LLM-Powered AI Agent Framework for Holistic IoT Traffic Interpretationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems
  2. 2025CoT-RAG: Integrating Chain of Thought and Retrieval-Augmented Generation to Enhance Reasoning in Large Language Models
  3. 2025Enhancing Large Language Models (LLMs) for Telecommunications using Knowledge Graphs and Retrieval-Augmented Generation
  4. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs
  5. 2026RFCLLM: Evaluating LLMs' Reasoning Ability of Network Protocol State Machines
  6. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
  7. 2026Tiny-Critic RAG: Empowering Agentic Fallback with Parameter-Efficient Small Language Models
  8. 2026Is Agentic RAG worth it? An experimental comparison of RAG approaches