mlsanityData to 14 Oct 2026

Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Bases

arXiv 2410.14594 ↗First posted 18 Oct 2024cs.CLcited by 23

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Recent advancements in tool-equipped Agents (LLMs) have enabled complex tasks like secure database interactions and multi-agent code development. However, scaling tool capacity beyond agent reasoning or model limits remains a challenge. In this paper, we address these challenges by introducing Toolshed Knowledge Bases, a tool knowledge base (vector database) designed to store enhanced tool representations and optimize tool selection for large-scale tool-equipped Agents. Additionally, we propose Advanced RAG-Tool Fusion, a novel ensemble of tool-applied advanced retrieval-augmented generation (RAG) techniques across the pre-retrieval, intra-retrieval, and post-retrieval phases, without requiring model fine-tuning. During pre-retrieval, tool documents are enhanced with key information and stored in the Toolshed Knowledge Base. Intra-retrieval focuses on query planning and transformation to increase retrieval accuracy. Post-retrieval refines the retrieved tool documents and enables self-reflection. Furthermore, by varying both the total number of tools (tool-M) an Agent has access to and the tool selection threshold (top-k), we address trade-offs between retrieval accuracy, agent performance, and token cost. Our approach achieves 46%, 56%, and 47% absolute improvements on the ToolE single-tool, ToolE multi-tool and Seal-Tools benchmark datasets, respectively (Recall@5).

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 42 of its 54 references that are in the map. First posted on arXiv 18 Oct 2024.

Builds on

42 of 54 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024CITI: Enhancing Tool Utilizing Ability in Large Language Models without Sacrificing General Performance20 Sep
  2. 2024Efficient and Scalable Estimation of Tool Representations in Vector Space2 Sep
  3. 2024ToolACE: Winning the Points of LLM Function Calling2 Sep
  4. 2024Tool Learning with Large Language Models: A Survey28 May
  5. 2024Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmark14 May
  6. 2024Planning and Editing What You Retrieve for Enhanced Tool Learning30 Mar
  7. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls6 Feb
  8. 2024EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction11 Jan
  9. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use4 Oct
  10. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  11. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases8 Jun
  12. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
2 more
  1. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings19 May
  2. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr

Other references · 28

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  6. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  7. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  8. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  9. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  10. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 360
  11. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
  12. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
16 more
  1. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 257
  2. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
  3. 2024Corrective Retrieval Augmented Generationcited by 197
  4. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Modelscited by 184
  5. 2023Take a Step Back: Evoking Reasoning via Abstraction in Large Language Modelscited by 159
  6. 2023Query2doc: Query Expansion with Large Language Modelscited by 149
  7. 2023CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsetscited by 129
  8. 2024Blended RAG: Improving RAG (Retriever-Augmented Generation) Accuracy with Semantic Search and Hybrid Query-Based Retrieverscited by 84
  9. 2023Large Language Model based Long-tail Query Rewriting in Taobao Searchcited by 62
  10. 2023Query Expansion by Prompting Large Language Modelscited by 57
  11. 2024Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrievalcited by 52
  12. 2024Improving Retrieval for RAG based Question Answering Models on Financial Documentscited by 49
  13. 2024ToolRerank: Adaptive and Hierarchy-Aware Reranking for Tool Retrievalcited by 41
  14. 2024Towards Completeness-Oriented Tool Retrieval for Large Language Modelscited by 26
  15. 2024REAPER: Reasoning based Retrieval Planning for Complex RAG Systemscited by 16
  16. 2023ProTIP: Progressive Tool Retrieval Improves Planningcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 7.

23 papers.

Sort:

By area:

  1. 2025OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoningcited by 69
  2. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 62
  3. 2025ScaleMCP: Dynamic and Auto-Synchronizing Model Context Protocol Tools for LLM Agentscited by 36
  4. 2025MemTool: Optimizing Short-Term Memory Management for Dynamic Tool Calling in LLM Agent Multi-Turn Conversationscited by 21
  5. 2025ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filteringcited by 19
  6. 2026Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Taskscited by 19
  7. 2025LongFuncEval: Measuring the effectiveness of long context models for function callingcited by 18
  8. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 16
  9. 2025Graph RAG-Tool Fusioncited by 15
  10. 2025Tool-to-Agent Retrieval: Bridging Tools and Agents for Scalable LLM Multi-Agent Systemscited by 15
  11. 2025Comparison of Text-Based and Image-Based Retrieval in Multimodal Retrieval Augmented Generation Large Language Model Systemscited by 5
  12. 2025Agent-as-a-Graph: Knowledge Graph-Based Tool and Agent Retrieval for LLM Multi-Agent Systemscited by 4
11 more
  1. 2025Solving Context Window Overflow in AI Agentscited by 4
  2. 2026Beyond Rows to Reasoning: Agentic Retrieval for Multimodal Spreadsheet Understanding and Editingcited by 4
  3. 2025Online-Optimized RAG for Tool Use and Function Callingcited by 2
  4. 2025Rethinking Retrieval: From Traditional Retrieval Augmented Generation to Agentic and Non-Vector Reasoning Systems in the Financial Domain for Large Language Modelscited by 2
  5. 2025In-N-Out: A Parameter-Level API Graph Dataset for Tool Agentscited by 1
  6. 2026Don't Offer What Can't Be Done: Deterministic Executability Gating for LLM Skill Selection at Scalecited by 1
  7. 2025Repairing Tool Calls Using Post-tool Execution Reflection and RAGcited by 0
  8. 2025ETOM: A Five-Level Benchmark for Evaluating Tool Orchestration within the MCP Ecosystemcited by 0
  9. 2026Agentic Jackal: Live Execution and Semantic Value Grounding for Text-to-JQLcited by 0
  10. 2026Adaptive Query Routing: A Tier-Based Framework for Hybrid Retrieval Across Financial, Legal, and Medical Documentscited by 0
  11. 2026CoHyDE: Iterative Co-Training of LLM Rewriter & Dense Encoder for Tool Retrievalcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Tools Are Not Islands: Set-Level Tool Retrieval for LLM Agents via Query-Conditioned Hyperedge Prediction
  2. 2026Beyond Single-Shot: Multi-step Tool Retrieval via Query Planning
  3. 2026Extracting Knowledge from Tools in LLM Agents
  4. 2025GRETEL: A Goal-driven Retrieval and Execution-based Trial Framework for LLM Tool Selection Enhancing
  5. 2025Keyword search is all you need: Achieving RAG-Level Performance without vector databases using agentic tool use
  6. 2026A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces
  7. 2026SchemaRouter: Field-Aware Tool Routing for Efficient Heterogeneous Agentic RAG
  8. 2026ToolFlood: Beyond Selection -- Hiding Valid Tools from LLM Agents via Semantic Covering