# Butterfly Effects in Toolchains: A Comprehensive Analysis of Failed Parameter Filling in LLM Tool-Agent Systems

arXiv: https://arxiv.org/abs/2507.15296 · HTML: https://mlsanity.com/paper/2507.15296/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 21 Jul 2025.

## Placed in

- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 16 of 16)

- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs](https://mlsanity.com/paper/2304.08244/) (2023)
- [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023)
- [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024)
- [ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases](https://mlsanity.com/paper/2306.05301/) (2023)
- [Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents](https://mlsanity.com/paper/2410.02644/) (2024)
- [ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities](https://mlsanity.com/paper/2408.04682/) (2024)
- [MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use](https://mlsanity.com/paper/2310.03128/) (2023)
- [LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples](https://mlsanity.com/paper/2310.01469/) (2023)
- [Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios](https://mlsanity.com/paper/2401.17167/) (2024)
- [ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages](https://mlsanity.com/paper/2402.10753/) (2024)
- [GTA: A Benchmark for General Tool Agents](https://mlsanity.com/paper/2407.08713/) (2024)
- [Crabs: Consuming Resource via Auto-generation for LLM-DoS Attack under Black-box Settings](https://mlsanity.com/paper/2412.13879/) (2024)
- [Evaluating Tool-Augmented Agents in Remote Sensing Platforms](https://mlsanity.com/paper/2405.00709/) (2024)

## Built on by (most cited first, 18 of 18)

- [AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition](https://mlsanity.com/paper/2602.11348/) (2026)
- [The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration](https://mlsanity.com/paper/2603.22862/) (2026)
- [Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures](https://mlsanity.com/paper/2607.28802/) (2026)
- [Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning](https://mlsanity.com/paper/2509.06461/) (2025)
- [Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments](https://mlsanity.com/paper/2605.27209/) (2026)
- [ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibration](https://mlsanity.com/paper/2601.06860/) (2026)
- [Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use](https://mlsanity.com/paper/2607.01084/) (2026)
- [Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in Large Language Models](https://mlsanity.com/paper/2508.01554/) (2025)
- [Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents](https://mlsanity.com/paper/2510.17491/) (2025)
- [PRAXIS: Integrating Program Analysis with Observability for Root-Cause Analysis](https://mlsanity.com/paper/2512.22113/) (2025)
- [LLM-enabled Applications Require System-Level Threat Monitoring](https://mlsanity.com/paper/2602.19844/) (2026)
- [Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling](https://mlsanity.com/paper/2609.00949/) (2026)
- [When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors](https://mlsanity.com/paper/2604.21255/) (2026)
- [Emerging from Ground: Addressing Intent Deviation in Tool-Using Agents via Deriving Real Calls into Virtual Trajectories](https://mlsanity.com/paper/2601.15120/) (2026)
- [PredAct-Bench: Benchmarking Tool-Augmented Dialogue under Controlled Tool Noise](https://mlsanity.com/paper/2608.02372/) (2026)
- [Using Grounded Theory for Agent Behavior Analysis at Scale](https://mlsanity.com/paper/2608.30391/) (2026)
- [What Stops a Small Language Model From Driving a Database Agent](https://mlsanity.com/paper/2609.21341/) (2026)
- [Understanding and Mitigating Hallucination Escape in Tool-Using LLM Agents](https://mlsanity.com/paper/2610.04409/) (2026)

## Abstract (arXiv)

The emergence of the tool agent paradigm has broadened the capability boundaries of the Large Language Model (LLM), enabling it to complete more complex tasks. However, the effectiveness of this paradigm is limited due to the issue of parameter failure during its execution. To explore this phenomenon and propose corresponding suggestions, we first construct a parameter failure taxonomy in this paper. We derive five failure categories from the invocation chain of a mainstream tool agent. Then, we explore the correlation between three different input sources and failure categories by applying 15 input perturbation methods to the input. Experimental results show that parameter name hallucination failure primarily stems from inherent LLM limitations, while issues with input sources mainly cause other failure patterns. To improve the reliability and effectiveness of tool-agent interactions, we propose corresponding improvement suggestions, including standardizing tool return formats, improving error feedback mechanisms, and ensuring parameter consistency.