# Tool-Planner: Task Planning with Clusters across Multiple Tools

arXiv: https://arxiv.org/abs/2406.03807 · HTML: https://mlsanity.com/paper/2406.03807/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 52 of its 59 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 30 of 52)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [ProgPrompt: Generating Situated Robot Task Plans using Large Language Models](https://mlsanity.com/paper/2209.11302/) (2022)
- [Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language](https://mlsanity.com/paper/2204.00598/) (2022)
- [Augmented Language Models: a Survey](https://mlsanity.com/paper/2302.07842/) (2023)
- [Complexity-Based Prompting for Multi-Step Reasoning](https://mlsanity.com/paper/2210.00720/) (2022)
- [Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models](https://mlsanity.com/paper/2304.09842/) (2023)
- [Large Language Models as Tool Makers](https://mlsanity.com/paper/2305.17126/) (2023)

## Built on by (most cited first, 30 of 33)

- [AutoData: A Multi-Agent System for Open Web Data Collection](https://mlsanity.com/paper/2505.15859/) (2025)
- [ToolTree: Efficient LLM Agent Tool Planning via Dual-Feedback Monte Carlo Tree Search and Bidirectional Pruning](https://mlsanity.com/paper/2603.12740/) (2026)
- [A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications](https://mlsanity.com/paper/2605.07358/) (2026)
- [ToolGate: Contract-Grounded and Verified Tool Execution for LLMs](https://mlsanity.com/paper/2601.04688/) (2026)
- [Towards Reliable Benchmarking: A Contamination Free, Controllable Evaluation Framework for Multi-step LLM Function Calling](https://mlsanity.com/paper/2509.26553/) (2025)
- [GTool: Graph Enhanced Tool Planning with Large Language Model](https://mlsanity.com/paper/2508.12725/) (2025)
- [AutoTool: Efficient Tool Selection for Large Language Model Agents](https://mlsanity.com/paper/2511.14650/) (2025)
- [Routine: A Structural Planning Framework for LLM Agent System in Enterprise](https://mlsanity.com/paper/2507.14447/) (2025)
- [PhGPO: Pheromone-Guided Policy Optimization for Long-Horizon Tool Planning](https://mlsanity.com/paper/2602.13691/) (2026)
- [EvoMem: Improving Multi-Agent Planning with Dual-Evolving Memory](https://mlsanity.com/paper/2511.01912/) (2025)
- [The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Check](https://mlsanity.com/paper/2601.12979/) (2026)
- [On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows](https://mlsanity.com/paper/2504.01931/) (2025)
- [PEToolLLM: Towards Personalized Tool Learning in Large Language Models](https://mlsanity.com/paper/2502.18980/) (2025)
- [NaviAgent: Graph-Driven Bilevel Planning for Scalable Tool Orchestration](https://mlsanity.com/paper/2506.19500/) (2025)
- [NC2C: Automated Convexification of Generic Non-Convex Optimization Problems](https://mlsanity.com/paper/2601.04789/) (2026)
- [TopoPilot: Reliable Conversational Workflow Automation for Topological Data Analysis and Visualization](https://mlsanity.com/paper/2603.25063/) (2026)
- [DaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agents](https://mlsanity.com/paper/2510.19336/) (2025)
- [From Instruction to Event: Sound-Triggered Mobile Manipulation](https://mlsanity.com/paper/2601.21667/) (2026)
- [OrchDAG: Complex Tool Orchestration in Multi-Turn Interactions with Plan DAGs](https://mlsanity.com/paper/2510.24663/) (2025)
- [PEARL: Plan Exploration and Adaptive Reinforcement Learning for Multihop Tool Use](https://mlsanity.com/paper/2601.20439/) (2026)
- [SWI: Speaking with Intent in Large Language Models](https://mlsanity.com/paper/2503.21544/) (2025)
- [Bridging Tool Dependencies and Domain Knowledge: A Graph-Based Framework for In-Context Planning](https://mlsanity.com/paper/2510.24690/) (2025)
- [ToolWeaver: Weaving Collaborative Semantics for Scalable Tool Use in Large Language Models](https://mlsanity.com/paper/2601.21947/) (2026)
- [PAC-CF: Calibrating Irreversible Frontier Pruning in LLM-Guided Search](https://mlsanity.com/paper/2604.14345/) (2026)
- [FitText: Evolving Agent Tool Ecologies via Memetic Retrieval](https://mlsanity.com/paper/2605.02411/) (2026)
- [VATS: Exploiting Implicit Authority in Error-Path Injection via Systematic Mutation](https://mlsanity.com/paper/2606.07992/) (2026)
- [Agents in the Wild: Where Research Meets Deployment](https://mlsanity.com/paper/2607.19336/) (2026)
- [Towards Trustworthy and Cost-Efficient Data Integration: From Na"ive RAG to Agentic RAG](https://mlsanity.com/paper/2607.22319/) (2026)
- [ToolLIFT: Lifting Tool-Specific Trajectories into Function-Level Graphs for Generalizable Tool Planning](https://mlsanity.com/paper/2608.03468/) (2026)
- [VTO: Visual Tool Orchestration for Video Anomaly Detection](https://mlsanity.com/paper/2608.08219/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated exceptional reasoning capabilities, enabling them to solve various complex problems. Recently, this ability has been applied to the paradigm of tool learning. Tool learning involves providing examples of tool usage and their corresponding functions, allowing LLMs to formulate plans and demonstrate the process of invoking and executing each tool. LLMs can address tasks that they cannot complete independently, thereby enhancing their potential across different tasks. However, this approach faces two key challenges. First, redundant error correction leads to unstable planning and long execution time. Additionally, designing a correct plan among multiple tools is also a challenge in tool learning. To address these issues, we propose Tool-Planner, a task-processing framework based on toolkits. Tool-Planner groups tools based on the API functions with the same function into a toolkit and allows LLMs to implement planning across the various toolkits. When a tool error occurs, the language model can reselect and adjust tools based on the toolkit. Experiments show that our approach demonstrates a high pass and win rate across different datasets and optimizes the planning scheme for tool learning in models such as GPT-4 and Claude 3, showcasing the potential of our method. Our code is public at https://github.com/OceannTwT/Tool-Planner