ChainFuzzer: Greybox Fuzzing for Workflow-Level Multi-Tool Vulnerabilities in LLM Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
Tool-augmented LLM agents increasingly rely on multi-step, multi-tool workflows to complete real tasks. This design expands the attack surface, because data produced by one tool can be persisted and later reused as input to another tool, enabling exploitable source-to-sink dataflows that only emerge through tool composition. We study this risk as multi-tool vulnerabilities in LLM agents, and show that existing discovery efforts focused on single-tool or single-hop testing miss these long-horizon behaviors and provide limited debugging value. We present ChainFuzzer, a greybox framework for discovering and reproducing multi-tool vulnerabilities with auditable evidence. ChainFuzzer (i) identifies high-impact operations with strict source-to-sink dataflow evidence and extracts plausible upstream candidate tool chains based on cross-tool dependencies, (ii) uses Trace-guided Prompt Solving (TPS) to synthesize stable prompts that reliably drive the agent to execute target chains, and (iii) performs guardrail-aware fuzzing to reproduce vulnerabilities under LLM guardrails via payload mutation and sink-specific oracles. We evaluate ChainFuzzer on 20 popular open-source LLM agent apps (998 tools). ChainFuzzer extracts 2,388 candidate tool chains and synthesizes 2,213 stable prompts, confirming 365 unique, reproducible vulnerabilities across 19/20 apps (302 require multi-tool execution). Component evaluation shows tool-chain extraction achieves 96.49% edge precision and 91.50% strict chain precision; TPS increases chain reachability from 27.05% to 95.45%; guardrail-aware fuzzing boosts payload-level trigger rate from 18.20% to 88.60%. Overall, ChainFuzzer achieves 3.02 vulnerabilities per 1M tokens, providing a practical foundation for testing and hardening real-world multi-tool agent systems.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 15 of its 49 references that are in the map. First posted on arXiv 13 Mar 2026.
Builds on
15 of 49 references.
Closest ancestors · 4
Same direction, newest first
- 2026Agent-Fence: Mapping Security Vulnerabilities Across Deep Research Agents7 Feb
- 2025A Survey on Agentic Security: Applications, Threats and Defenses7 Oct
- 2025MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers19 Aug
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
Other references · 11
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2024LLM Dataset Inference: Did you train on my dataset?cited by 110
- 2018TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzingcited by 68
- 2023From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?cited by 55
- 2025Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection Learningcited by 24
- 2024Evaluation of LLM Vulnerabilities to Being Misused for Personalized Disinformation Generationcited by 20
- 2023Harnessing the Power of LLM to Support Binary Taint Analysiscited by 13
- 2022DCDetector: An IoT terminal vulnerability mining system based on distributed deep ensemble learning under source code representationcited by 1
- 2023Security Weaknesses in IoT Management Platformscited by 1
- 2025Demystifying Cookie Sharing Risks in WebView-based Mobile App-in-app Ecosystemscited by 1
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agentscited by 18
- 2026ChainCaps: Composition-Safe Tool-Using Agents via Monotonic Capability Attenuationcited by 11
- 2026SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skillscited by 5
- 2026AgentFlow: Building Agent Dependency Graphs for Static Analysis of Agent Programscited by 2
- 2026Tangent: An Empirical Study of Testing Practices for LLM-Based Agent Applicationscited by 2
- 2026Demystifying and Detecting Agentic Workflow Injection Vulnerabilities in GitHub Actionscited by 1
- 2026VIPER-MCP: Detecting and Exploiting Taint-Style Vulnerabilities in Model Context Protocol Serverscited by 0
- 2026Securing Multi-Tool AI Agent Chains With Dynamic, Real-Time Compositional Policiescited by 0
- 2026G-CARB: Graph-Localized Conformal Agent Risk Budget for Compositional Harmcited by 0
- 2026G-CARB: Graph-Localized Conformal Agent Risk Budget for Compositional Harm4 Oct 2026
- 2026Tangent: An Empirical Study of Testing Practices for LLM-Based Agent Applications9 Aug 2026
- 2026Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agents11 Jul 2026
- 2026Securing Multi-Tool AI Agent Chains With Dynamic, Real-Time Compositional Policies3 Jul 2026
- 2026AgentFlow: Building Agent Dependency Graphs for Static Analysis of Agent Programs2 Jul 2026
- 2026ChainCaps: Composition-Safe Tool-Using Agents via Monotonic Capability Attenuation26 May 2026
- 2026VIPER-MCP: Detecting and Exploiting Taint-Style Vulnerabilities in Model Context Protocol Servers20 May 2026
- 2026Demystifying and Detecting Agentic Workflow Injection Vulnerabilities in GitHub Actions8 May 2026
- 2026SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skills7 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026FlowSteer: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systems
- 2026FuzzingBrain V2: A Multi-Agent LLM System for Automated Vulnerability Discovery and Reproduction
- 2025ToolTweak: An Attack on Tool Selection in LLM-based Agents
- 2025Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents
- 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
- 2026SeedSmith: LLM-Driven Seed Synthesis for Directed Fuzzing ↗
- 2026MalTool: Malicious Tool Attacks on LLM Agents
- 2026Guardrails as Scapegoats: Auditing Unfaithful Safety Refusals in Tool-Augmented LLM Agents