Tools are under-documented: Simple Document Expansion Boosts Tool Retrieval
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) have recently demonstrated strong capabilities in tool use, yet progress in tool retrieval remains hindered by incomplete and heterogeneous tool documentation. To address this challenge, we introduce Tool-DE, a new benchmark and framework that systematically enriches tool documentation with structured fields to enable more effective tool retrieval, together with two dedicated models, Tool-Embed and Tool-Rank. We design a scalable document expansion pipeline that leverages both open- and closed-source LLMs to generate, validate, and refine enriched tool profiles at low cost, producing large-scale corpora with 50k instances for embedding-based retrievers and 200k for rerankers. On top of this data, we develop two models specifically tailored for tool retrieval: Tool-Embed, a dense retriever, and Tool-Rank, an LLM-based reranker. Extensive experiments on ToolRet and Tool-DE demonstrate that document expansion substantially improves retrieval performance, with Tool-Embed and Tool-Rank achieving new state-of-the-art results on both benchmarks. We further analyze the contribution of individual fields to retrieval effectiveness, as well as the broader impact of document expansion on both training and evaluation. Overall, our findings highlight both the promise and limitations of LLM-driven document expansion, positioning Tool-DE, along with the proposed Tool-Embed and Tool-Rank, as a foundation for future research in tool retrieval.
Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Oct 2025.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026From Skill Text to Skill Structure: The Scheduling-Structural-Logical Representation for Agent Skillscited by 16
- 2026Beyond Single-Shot: Multi-step Tool Retrieval via Query Planningcited by 3
- 2026UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agentscited by 3
- 2026Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrievalcited by 2
- 2026The Menu Is an Execution Prior: State-Path Tool Menus for Online Agentscited by 2
- 2026AgentSearchBench: A Benchmark for AI Agent Search in the Wildcited by 1
- 2026Scaling Enterprise Agent Routing: Degradation, Diagnosis, and Recoverycited by 1
- 2026MagicSelector: Joint Optimization for Agent Tool Selection via Counterfactual Decomposition and Progressive Rerankingcited by 1
- 2026SE-GoS: Self-Evolving Graph-of-Skills for Skill Library at Scalecited by 1
- 2026SkillSeek: Revisiting Agent Skill Retrieval at Marketplace Scalecited by 1
- 2026CompRank: Efficient LLM Reranking via Token-Level Compression and Decoding-Free Scoringcited by 0
- 2026Enrich-Retrieve-Rank: Scaling Capability Discovery Beyond In-Context Routingcited by 0
- 2026SkillSeek: Revisiting Agent Skill Retrieval at Marketplace Scale30 Sep 2026
- 2026The Menu Is an Execution Prior: State-Path Tool Menus for Online Agents8 Sep 2026
- 2026SE-GoS: Self-Evolving Graph-of-Skills for Skill Library at Scale8 Sep 2026
- 2026Enrich-Retrieve-Rank: Scaling Capability Discovery Beyond In-Context Routing24 Aug 2026
- 2026MagicSelector: Joint Optimization for Agent Tool Selection via Counterfactual Decomposition and Progressive Reranking20 Jul 2026
- 2026Scaling Enterprise Agent Routing: Degradation, Diagnosis, and Recovery16 Jun 2026
- 2026CompRank: Efficient LLM Reranking via Token-Level Compression and Decoding-Free Scoring10 Jun 2026
- 2026From Skill Text to Skill Structure: The Scheduling-Structural-Logical Representation for Agent Skills27 Apr 2026
- 2026AgentSearchBench: A Benchmark for AI Agent Search in the Wild24 Apr 2026
- 2026UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agents13 Apr 2026
- 2026Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrieval1 Mar 2026
- 2026Beyond Single-Shot: Multi-step Tool Retrieval via Query Planning12 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models
- 2024Efficient and Scalable Estimation of Tool Representations in Vector Space
- 2024Enhancing Tool Retrieval with Iterative Feedback from Large Language Models
- 2024ToolRerank: Adaptive and Hierarchy-Aware Reranking for Tool Retrieval
- 2024Towards Completeness-Oriented Tool Retrieval for Large Language Models
- 2026Semantic Tool Discovery for Large Language Models: A Vector-Based Approach to MCP Tool Selection
- 2026ToolWeaver: Weaving Collaborative Semantics for Scalable Tool Use in Large Language Models
- 2024Data-Efficient Massive Tool Retrieval: A Reinforcement Learning Approach for Query-Tool Alignment with Language Models