mlsanityData to 14 Oct 2026

When Tools Get in the Way: The Effect of Unnecessary Tool Availability on LLM Answering

arXiv 2609.14157 ↗First posted 12 Sep 2026cs.CLcited by 0

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Large language models (LLMs) are increasingly deployed with external tools that extend what they can do beyond their own knowledge. Tools help on tasks that need external information, but their availability may also change how a model handles questions that do not need them. Prior work has mostly asked whether models select and use tools appropriately; whether an unnecessary tool changes the correctness of answers has received less attention. We ask whether making a related but unnecessary tool available affects a model's ability to answer from its own knowledge, and whether a preceding tool interaction changes this behaviour. We construct 500 query pairs across 10 knowledge domains. Each pair consists of a tool query, which needs the domain's tool, and a closed-domain query, which does not. Six LLMs are evaluated with the tool unavailable, available, and available after a prior tool call. Across 3,000 baseline trials the pooled answer rate is 98.2%. When an unnecessary tool is available it falls to 63.5%, with large differences between models. The decrease occurs even when the tool is rarely called, so it cannot be explained by unnecessary tool invocation alone. A one-sentence scope-aware system instruction recovers most of the lost answers.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 23 of its 27 references that are in the map. First posted on arXiv 12 Sep 2026.

Builds on

23 of 27 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use13 May
  2. 2026LLM Agents Already Know When to Call Tools -- Even Without Reasoning10 May
  3. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use4 Oct
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  5. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb

Other references · 17

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  4. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  5. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  6. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  7. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  8. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  9. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
  10. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
  11. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 145
  12. 2025FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoningcited by 58
5 more
  1. 2025Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Modelscited by 10
  2. 2026The Causal Impact of Tool Affordance on Safety Alignment in LLM Agentscited by 7
  3. 2026The Tool-Overuse Illusion: Why Does LLM Prefer External Tools over Internal Knowledge?cited by 6
  4. 2026Are LLM Evaluators Really Narcissists? Sanity Checking Self-Preference Evaluationscited by 5
  5. 2026Do LLMs Know Tool Irrelevance? Demystifying Structural Alignment Bias in Tool Invocationscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Can Tool-augmented Large Language Models be Aware of Incomplete Conditions?
  2. 2025AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarification
  3. 2023ToolQA: A Dataset for LLM Question Answering with External Tools
  4. 2025Tool Preferences in Agentic LLMs are Unreliable
  5. 2025LongFuncEval: Measuring the effectiveness of long context models for function calling
  6. 2024Learning to Ask: When LLM Agents Meet Unclear Instruction
  7. 2024ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios
  8. 2025ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filtering