# Efficient and Scalable Estimation of Tool Representations in Vector Space

arXiv: https://arxiv.org/abs/2409.02141 · HTML: https://mlsanity.com/paper/2409.02141/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use.
- Builds on 34 of its 41 references that are in the map.
- First posted on arXiv 2 Sep 2024.

## Placed in

- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 30 of 34)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs](https://mlsanity.com/paper/2304.08244/) (2023)
- [ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases](https://mlsanity.com/paper/2306.05301/) (2023)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [Large Language Models as Tool Makers](https://mlsanity.com/paper/2305.17126/) (2023)
- [ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings](https://mlsanity.com/paper/2305.11554/) (2023)
- [MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use](https://mlsanity.com/paper/2310.03128/) (2023)
- [An LLM Compiler for Parallel Function Calling](https://mlsanity.com/paper/2312.04511/) (2023)
- [MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning](https://mlsanity.com/paper/2205.00445/) (2022)
- [EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction](https://mlsanity.com/paper/2401.06201/) (2024)
- [Generalized Planning in PDDL Domains with Pretrained Large Language Models](https://mlsanity.com/paper/2305.11014/) (2023)
- [AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls](https://mlsanity.com/paper/2402.04253/) (2024)
- [Full Stack Optimization of Transformer Inference: a Survey](https://mlsanity.com/paper/2302.14017/) (2023)
- [LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement](https://mlsanity.com/paper/2403.15042/) (2024)
- [MM-LIMA: Less Is More for Alignment in Multi-Modal Datasets](https://mlsanity.com/paper/2308.12067/) (2023)
- [TinyAgent: Function Calling at the Edge](https://mlsanity.com/paper/2409.00608/) (2024)
- [Octopus v2: On-device language model for super agent](https://mlsanity.com/paper/2404.01744/) (2024)
- [ToolRerank: Adaptive and Hierarchy-Aware Reranking for Tool Retrieval](https://mlsanity.com/paper/2403.06551/) (2024)

## Built on by (most cited first, 22 of 22)

- [Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks](https://mlsanity.com/paper/2503.09572/) (2025)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [MCP-Zero: Active Tool Discovery for Autonomous LLM Agents](https://mlsanity.com/paper/2506.01056/) (2025)
- [Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Bases](https://mlsanity.com/paper/2410.14594/) (2024)
- [MCPVerse: An Expansive, Real-World Benchmark for Agentic Tool Use](https://mlsanity.com/paper/2508.16260/) (2025)
- [Toward Efficient Agents: Memory, Tool learning, and Planning](https://mlsanity.com/paper/2601.14192/) (2026)
- [Graph RAG-Tool Fusion](https://mlsanity.com/paper/2502.07223/) (2025)
- [ADC: Enhancing Function Calling Via Adversarial Datasets and Code Line-Level Feedback](https://mlsanity.com/paper/2412.17754/) (2024)
- [Agent-as-a-Graph: Knowledge Graph-Based Tool and Agent Retrieval for LLM Multi-Agent Systems](https://mlsanity.com/paper/2511.18194/) (2025)
- [SING: Synthetic Intention Graph for Scalable Active Tool Discovery in LLM Agents](https://mlsanity.com/paper/2606.16591/) (2026)
- [MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Models](https://mlsanity.com/paper/2507.00487/) (2025)
- [LoSemB: Logic-Guided Semantic Bridging for Inductive Tool Retrieval](https://mlsanity.com/paper/2508.07690/) (2025)
- [AgentSelect: Benchmark for Narrative Query-to-Agent Recommendation](https://mlsanity.com/paper/2603.03761/) (2026)
- [UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agents](https://mlsanity.com/paper/2604.11557/) (2026)
- [TabAgent: A Framework for Replacing Agentic Generative Components with Tabular-Textual Classifiers](https://mlsanity.com/paper/2602.16429/) (2026)
- [Concurrency without Model Changes: Future-based Asynchronous Function Calling for LLMs](https://mlsanity.com/paper/2605.15077/) (2026)
- [When Tool-Backed Skill Retrieval Fails: Source-Style Collapse in Executable Capability Retrieval](https://mlsanity.com/paper/2608.16502/) (2026)
- [Planning Agents on an Ego-Trip: Leveraging Hybrid Ego-Graph Ensembles for Improved Tool Retrieval in Enterprise Task Planning](https://mlsanity.com/paper/2508.05888/) (2025)
- [Try, Check and Retry: A Divide-and-Conquer Framework for Boosting Long-context Tool-Calling Performance of LLMs](https://mlsanity.com/paper/2603.11495/) (2026)
- [Non-negative Elastic Net Decoding for Information Retrieval](https://mlsanity.com/paper/2606.17910/) (2026)
- [FitText: Evolving Agent Tool Ecologies via Memetic Retrieval](https://mlsanity.com/paper/2605.02411/) (2026)
- [Self-Evolving Agents via Likelihood-Guided Tool-Space Optimization](https://mlsanity.com/paper/2609.34151/) (2026)

## Abstract (arXiv)

Recent advancements in function calling and tool use have significantly enhanced the capabilities of large language models (LLMs) by enabling them to interact with external information sources and execute complex tasks. However, the limited context window of LLMs presents challenges when a large number of tools are available, necessitating efficient methods to manage prompt length and maintain accuracy. Existing approaches, such as fine-tuning LLMs or leveraging their reasoning capabilities, either require frequent retraining or incur significant latency overhead. A more efficient solution involves training smaller models to retrieve the most relevant tools for a given query, although this requires high quality, domain-specific data. To address those challenges, we present a novel framework for generating synthetic data for tool retrieval applications and an efficient data-driven tool retrieval strategy using small encoder models. Empowered by LLMs, we create ToolBank, a new tool retrieval dataset that reflects real human user usages. For tool retrieval methodologies, we propose novel approaches: (1) Tool2Vec: usage-driven tool embedding generation for tool retrieval, (2) ToolRefiner: a staged retrieval method that iteratively improves the quality of retrieved tools, and (3) MLC: framing tool retrieval as a multi-label classification problem. With these new methods, we achieve improvements of up to 27.28 in Recall@K on the ToolBench dataset and 30.5 in Recall@K on ToolBank. Additionally, we present further experimental results to rigorously validate our methods. Our code is available at urlhttps://github.com/SqueezeAILab/Tool2Vec