mlsanityData to 14 Oct 2026

Butterfly Effects in Toolchains: A Comprehensive Analysis of Failed Parameter Filling in LLM Tool-Agent Systems

arXiv 2507.15296 ↗First posted 21 Jul 2025cs.SE cs.AIcited by 18

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

The emergence of the tool agent paradigm has broadened the capability boundaries of the Large Language Model (LLM), enabling it to complete more complex tasks. However, the effectiveness of this paradigm is limited due to the issue of parameter failure during its execution. To explore this phenomenon and propose corresponding suggestions, we first construct a parameter failure taxonomy in this paper. We derive five failure categories from the invocation chain of a mainstream tool agent. Then, we explore the correlation between three different input sources and failure categories by applying 15 input perturbation methods to the input. Experimental results show that parameter name hallucination failure primarily stems from inherent LLM limitations, while issues with input sources mainly cause other failure patterns. To improve the reliability and effectiveness of tool-agent interactions, we propose corresponding improvement suggestions, including standardizing tool return formats, improving error feedback mechanisms, and ensuring parameter consistency.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 16 of its 34 references that are in the map. First posted on arXiv 21 Jul 2025.

Builds on

16 of 34 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages16 Feb
  2. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use4 Oct
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  4. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases8 Jun
  5. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
  6. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr

Other references · 10

  1. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
  2. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
  3. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 538
  4. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 369
  5. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
  6. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  7. 2024Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarioscited by 86
  8. 2024GTA: A Benchmark for General Tool Agentscited by 77
  9. 2024Crabs: Consuming Resource via Auto-generation for LLM-DoS Attack under Black-box Settingscited by 35
  10. 2024Evaluating Tool-Augmented Agents in Remote Sensing Platformscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 14.

18 papers.

Sort:

By area:

  1. 2026AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Conditioncited by 23
  2. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 14
  3. 2026Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failurescited by 10
  4. 2025Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoningcited by 8
  5. 2026Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environmentscited by 6
  6. 2026ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibrationcited by 4
  7. 2026Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Usecited by 4
  8. 2025Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in Large Language Modelscited by 3
  9. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  10. 2025PRAXIS: Integrating Program Analysis with Observability for Root-Cause Analysiscited by 2
  11. 2026LLM-enabled Applications Require System-Level Threat Monitoringcited by 2
  12. 2026Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Callingcited by 2
6 more
  1. 2026When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviorscited by 1
  2. 2026Emerging from Ground: Addressing Intent Deviation in Tool-Using Agents via Deriving Real Calls into Virtual Trajectoriescited by 0
  3. 2026PredAct-Bench: Benchmarking Tool-Augmented Dialogue under Controlled Tool Noisecited by 0
  4. 2026Using Grounded Theory for Agent Behavior Analysis at Scalecited by 0
  5. 2026What Stops a Small Language Model From Driving a Database Agentcited by 0
  6. 2026Understanding and Mitigating Hallucination Escape in Tool-Using LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ToolRobustBench: Stage-Wise Perturbation Evaluation and Failure Diagnosis for Tool-Calling Agents
  2. 2026Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures
  3. 2026Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents
  4. 2025More Vulnerable than You Think: On the Stability of Tool-Integrated LLM Agents
  5. 2026How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines
  6. 2025How Do LLMs Fail In Agentic Scenarios? A Qualitative Analysis of Success and Failure Scenarios of Various LLMs in Agentic Simulations
  7. 2026ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents
  8. 2026DOCSCHISEL: Adaptive Tool Documentation Optimization Framework for LLM Agents