# Do LLMs Know Tool Irrelevance? Demystifying Structural Alignment Bias in Tool Invocations

arXiv: https://arxiv.org/abs/2604.11322 · HTML: https://mlsanity.com/paper/2604.11322/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 36 references that are in the map.
- First posted on arXiv 13 Apr 2026.

## Builds on (most cited first, 22 of 22)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [The Power of Noise: Redefining Retrieval for RAG Systems](https://mlsanity.com/paper/2401.14887/) (2024)
- [Direct and Indirect Effects](https://mlsanity.com/paper/1301.2300/) (2013)
- [Retrieval Head Mechanistically Explains Long-Context Factuality](https://mlsanity.com/paper/2404.15574/) (2024)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [Evaluation and Benchmarking of LLM Agents: A Survey](https://mlsanity.com/paper/2507.21504/) (2025)
- [EmbeddingGemma: Powerful and Lightweight Text Representations](https://mlsanity.com/paper/2509.20354/) (2025)
- [Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2402.18154/) (2024)
- [Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculum](https://mlsanity.com/paper/2308.14034/) (2023)
- [LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions](https://mlsanity.com/paper/2509.18970/) (2025)
- [Induction Heads as an Essential Mechanism for Pattern Matching in In-context Learning](https://mlsanity.com/paper/2407.07011/) (2024)
- [Learning to Ask: When LLM Agents Meet Unclear Instruction](https://mlsanity.com/paper/2409.00557/) (2024)
- [When2Call: When (not) to Call Tools](https://mlsanity.com/paper/2504.18851/) (2025)
- [SafeToolBench: Pioneering a Prospective Benchmark to Evaluating Tool Utilization Safety in LLMs](https://mlsanity.com/paper/2509.07315/) (2025)
- [MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Tools](https://mlsanity.com/paper/2504.20168/) (2025)
- [Do not Abstain! Identify and Solve the Uncertainty](https://mlsanity.com/paper/2506.00780/) (2025)
- [Stochastic Chameleons: Irrelevant Context Hallucinations Reveal Class-Based (Mis)Generalization in LLMs](https://mlsanity.com/paper/2505.22630/) (2025)

## Built on by (most cited first, 3 of 3)

- [When Tools Get in the Way: The Effect of Unnecessary Tool Availability on LLM Answering](https://mlsanity.com/paper/2609.14157/) (2026)
- [MIRAGE: How Conversation State Shapes Historical Evidence Use in Multimodal Personal Agents](https://mlsanity.com/paper/2609.19059/) (2026)
- [One Task, Many Interfaces: Simple Cross-Interface Consistency Training Improves LLM Agent Robustness](https://mlsanity.com/paper/2610.20576/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated impressive capabilities in utilizing external tools. In practice, however, LLMs are often exposed to tools that are irrelevant to the user's query, in which case the desired behavior is to refrain from invocations. In this work, we identify a widespread yet overlooked mechanistic flaw in tool refusal, which we term structural alignment bias: Even when a tool fails to serve the user's goal, LLMs still tend to invoke it whenever query attributes can be validly assigned to tool parameters. To systematically study this bias, we introduce SABEval, a new dataset that decouples structural alignment from semantic relevance. Our analysis shows that structural alignment bias induces severe tool-invocation errors in LLMs, yet remains largely unaccounted for in existing evaluations. To investigate the internal mechanisms underlying this bias, we propose Contrastive Attention Attribution, which reveals two competing pathways for semantic checking and structural matching. The relative strength of these pathways drives LLMs' tool invocation decisions. Based on these findings, we further introduce a rebalancing strategy that effectively mitigates structural alignment bias, as demonstrated by extensive experiments, without degrading general tool-use capabilities.