When Tool-Backed Skill Retrieval Fails: Source-Style Collapse in Executable Capability Retrieval
Not on the map. Why?
Abstract (arXiv)
Large-scale agents increasingly rely on retrieval to access external capabilities. We study this retrieval gate in structured tools and APIs, a measurable class of tool-backed executable skills that must be surfaced before an agent can plan, incorporate, or act. In this setting the retrieval layer can silently fail even when the capability corpus is fixed: on ToolRet, a retriever fine-tuned on one source-specific slice collapses on another source-specific slice of the same benchmark, with FT-1100 despite its higher lexical overlap with the gold tools. We call this failure mode source-style collapse. Query-side TF-IDF fingerprints flag source styles on which the fine-tuned retriever is likely to fail better than semantic or length-based proxies, giving a cheap signal for mismatch over a fixed tool corpus. We propose ToolScout, a source-aware routing method that uses this signal as a routing guard: on the mixed 4,996-query stream, TF-IDF-based routing raises coverage from 22.3% to 86.1%, and across five collapsed sources 20 matched examples raise the coverage-weighted global top-1 proxy from 1.3% to 53.9%. The same failure and routing behaviors persist when tools are rerendered as executable skill cards, which rules out raw API-schema format as the sole cause.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 17 Aug 2026.
Builds on
25 of 33 references.
References · 25
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
- 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
13 more
- 2019Passage Re-ranking with BERTcited by 645
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
- 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 235
- 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agentscited by 130
- 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 124
- 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
- 2025DeepAgent: A General Reasoning Agent with Scalable Toolsetscited by 71
- 2025ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Usecited by 50
- 2026Skill Retrieval Augmentation for Agentic AIcited by 44
- 2024ToolRerank: Adaptive and Hierarchy-Aware Reranking for Tool Retrievalcited by 41
- 2026When Single-Agent with Skills Replace Multi-Agent Systems and When They Failcited by 35
- 2024Efficient and Scalable Estimation of Tool Representations in Vector Spacecited by 22
- 2022AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentationcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.