mlsanityData to 7 Oct 2026

ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings

arXiv 2305.11554 ↗First posted 19 May 2023cs.CL cs.LGcited by 238

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Augmenting large language models (LLMs) with external tools has emerged as a promising approach to solving complex problems. However, traditional methods, which finetune LLMs with tool demonstration data, can be both costly and restricted to a predefined set of tools. Recent in-context learning paradigm alleviates these issues, but the limited context length only allows for a few shots of demonstrations, leading to suboptimal understandings of the tools. Moreover, when there are numerous tools to choose from, in-context learning could completely fail to work. In this paper, we propose an alternative approach, ToolkenGPT, which combines the benefits of both sides. Our approach represents each underlinetool as a tounderlineken (textittoolken) and learns an embedding for it, enabling tool calls in the same way as generating a regular word token. Once a toolken is triggered, the LLM is prompted to complete arguments for the tool to execute. ToolkenGPT offers the flexibility to plug in an arbitrary number of tools by expanding the set of toolkens on the fly. In addition, it improves tool use by allowing extensive demonstration data for learning the toolken embeddings. In diverse domains, including numerical reasoning, knowledge-based question answering, and embodied plan generation, our approach effectively augments LLMs with tools and substantially outperforms various latest baselines. ToolkenGPT demonstrates the promising ability to use relevant tools from a large tool set in complex scenarios.

Key facts. Cited by 238 papers in the map (data through 7 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 62 of its 87 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

62 of 87 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Tool Learning with Foundation Models17 Apr
  2. 2023ART: Automatic multi-step reasoning and tool-use for large language models16 Mar
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb
  4. 2022TALM: Tool Augmented Language Models24 May

Other references · 58

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
38 more
  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  2. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  3. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  9. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  10. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  11. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  12. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  13. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  14. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  15. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  16. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  17. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  18. 2022PAL: Program-aided Language Modelscited by 635
  19. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  20. 2020Recipes for building an open-domain chatbotcited by 593
  21. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  22. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  23. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
  24. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  25. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  26. 2023Augmented Language Models: a Surveycited by 398
  27. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  28. 2023Faithful Chain-of-Thought Reasoningcited by 285
  29. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 220
  30. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  31. 2022Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Modelscited by 179
  32. 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 174
  33. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  34. 2023Translating Natural Language to Planning Goals with Large-Language Modelscited by 132
  35. 2023Solving Math Word Problems by Combining Language Models With Symbolic Solverscited by 124
  36. 2023Language Models Meet World Models: Embodied Experiences Enhance Language Modelscited by 123
  37. 2020A Survey of Knowledge-Enhanced Text Generationcited by 119
  38. 2023Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learningcited by 115

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 37; 2024: 97; 2025: 61; 2026: 43.

238 papers.

Sort:

By area:

  1. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  2. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  3. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  4. 2023A Comprehensive Overview of Large Language Modelscited by 639
  5. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  6. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  7. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  8. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  9. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  10. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  11. 2024Tool Learning with Large Language Models: A Surveycited by 221
  12. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
38 more
  1. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  2. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 155
  3. 2024EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instructioncited by 153
  4. 2024The BrowserGym Ecosystem for Web Agent Researchcited by 152
  5. 2024Continual Learning for Large Language Models: A Surveycited by 144
  6. 2024StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Modelscited by 129
  7. 2023CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsetscited by 122
  8. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 121
  9. 2024Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffoldingcited by 115
  10. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  11. 2023ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Searchcited by 103
  12. 2023T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Stepcited by 91
  13. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  14. 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agentscited by 89
  15. 2024Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarioscited by 85
  16. 2025StepSearch: Igniting LLMs Search Ability via Step-Wise Proximal Policy Optimizationcited by 82
  17. 2024Embodied LLM Agents Learn to Cooperate in Organized Teamscited by 81
  18. 2023Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arenacited by 78
  19. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  20. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agentscited by 77
  21. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 77
  22. 2023Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflowcited by 76
  23. 2024Unified Hallucination Detection for Multimodal Large Language Modelscited by 75
  24. 2024ToolGen: Unified Tool Retrieval and Calling via Generationcited by 72
  25. 2023Lemur: Harmonizing Natural Language and Code for Language Agentscited by 71
  26. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Modelscited by 70
  27. 2023Making Language Models Better Tool Learners with Execution Feedbackcited by 69
  28. 2024SciAgent: Tool-augmented Language Models for Scientific Reasoningcited by 69
  29. 2024Preble: Efficient Distributed Prompt Scheduling for LLM Servingcited by 68
  30. 2024MindSearch: Mimicking Human Minds Elicits Deep AI Searchercited by 65
  31. 2026Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scalecited by 65
  32. 2023Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculumcited by 64
  33. 2025AgentRxiv: Towards Collaborative Autonomous Researchcited by 64
  34. 2024Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmarkcited by 62
  35. 2024Yo'LLaVA: Your Personalized Language and Vision Assistantcited by 61
  36. 2024LLM With Tools: A Surveycited by 60
  37. 2023ControlLLM: Augment Language Models with Tools by Searching on Graphscited by 59
  38. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planningcited by 59

50 most cited of 238 All 238 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Flexibly Scaling Large Language Models Contexts Through Extensible Tokenization
  2. 2026Bootstrapping Embeddings for Low Resource Languages
  3. 2024BGE Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models
  4. 2025ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool Learning
  5. 2025Self-Training Large Language Models for Tool-Use Without Demonstrations
  6. 2025Chain-of-Tools: Utilizing Massive Unseen Tools in the CoT Reasoning of Frozen Language Models
  7. 2025Improving Multilingual Language Models by Aligning Representations through Steering
  8. 2024WTU-EVAL: A Whether-or-Not Tool Usage Evaluation Benchmark for Large Language Models