mlsanityData to 7 Oct 2026

ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search

arXiv 2310.13227 ↗First posted 20 Oct 2023cs.CL cs.AI cs.LGcited by 103

Not on the map. Why?

Cited by 60 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have demonstrated powerful decision-making and planning capabilities in solving complicated real-world problems. LLM-based autonomous agents can interact with diverse tools (e.g., functional APIs) and generate solution plans that execute a series of API function calls in a step-by-step manner. The multitude of candidate API function calls significantly expands the action space, amplifying the critical need for efficient action space navigation. However, existing methods either struggle with unidirectional exploration in expansive action spaces, trapped into a locally optimal solution, or suffer from exhaustively traversing all potential actions, causing inefficient navigation. To address these issues, we propose ToolChain*, an efficient tree search-based planning algorithm for LLM-based agents. It formulates the entire action space as a decision tree, where each node represents a possible API function call involved in a solution plan. By incorporating the A* search algorithm with task-specific cost function design, it efficiently prunes high-cost branches that may involve incorrect actions, identifying the most low-cost valid path as the solution. Extensive experiments on multiple tool-use and reasoning tasks demonstrate that ToolChain* efficiently balances exploration and exploitation within an expansive action space. It outperforms state-of-the-art baselines on planning and reasoning tasks by 3.1% and 3.5% on average while requiring 7.35x and 2.31x less time, respectively.

Key facts. Cited by 103 papers in the map (data through 7 Oct 2026). Builds on 44 of its 60 references that are in the map. First posted on arXiv 20 Oct 2023.

Builds on

44 of 60 references.

References · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  12. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
32 more
  1. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  2. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  3. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  4. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  5. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  6. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  7. 2023PaLM 2 Technical Reportcited by 1,154
  8. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  9. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  10. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  11. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  12. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  13. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  14. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  15. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  16. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  17. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  18. 2023Language Models can Solve Computer Taskscited by 371
  19. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  20. 2023Tool Learning with Foundation Modelscited by 302
  21. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 245
  22. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  23. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  24. 2022TALM: Tool Augmented Language Modelscited by 197
  25. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 195
  26. 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 193
  27. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  28. 2023On the Tool Manipulation Capability of Open-source Large Language Modelscited by 122
  29. 2023GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Informationcited by 115
  30. 2023Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPTcited by 78
  31. 2022Solving Math Word Problems via Cooperative Reasoning induced Language Modelscited by 76
  32. 2023Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modelingcited by 56

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 41; 2025: 32; 2026: 30.

103 papers.

Sort:

By area:

  1. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  2. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  3. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
  4. 2024Tool Learning with Large Language Models: A Surveycited by 221
  5. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  6. 2024EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instructioncited by 153
  7. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
  8. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  9. 2024G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networkscited by 110
  10. 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learningcited by 107
  11. 2025Multi-Agent Collaboration via Evolving Orchestrationcited by 102
  12. 2024GUI Agents: A Surveycited by 101
38 more
  1. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planningcited by 95
  2. 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilitiescited by 82
  3. 2024EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Recordscited by 75
  4. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Modelscited by 70
  5. 2024HYDRA: Model Factorization Framework for Black-Box LLM Personalizationcited by 59
  6. 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 56
  7. 2024Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agentscited by 56
  8. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  9. 2024From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactionscited by 46
  10. 2024Cocoa: Co-Planning and Co-Execution with AI Agentscited by 38
  11. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  12. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  13. 2024System-1.x: Learning to Balance Fast and Slow Planning with Language Modelscited by 35
  14. 2024Efficient Tool Use with Chain-of-Abstraction Reasoningcited by 33
  15. 2024Tool-Planner: Task Planning with Clusters across Multiple Toolscited by 32
  16. 2024Non-myopic Generation of Language Models for Reasoning and Planningcited by 25
  17. 2024Security of AI Agentscited by 24
  18. 2024A Trembling House of Cards? Mapping Adversarial Attacks against Language Agentscited by 22
  19. 2025A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworkscited by 21
  20. 2024DeLLMa: Decision Making Under Uncertainty with Large Language Modelscited by 20
  21. 2025Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environmentscited by 19
  22. 2026ToolTree: Efficient LLM Agent Tool Planning via Dual-Feedback Monte Carlo Tree Search and Bidirectional Pruningcited by 18
  23. 2024Devil's Advocate: Anticipatory Reflection for LLM Agentscited by 17
  24. 2025Divide-Then-Aggregate: An Efficient Tool Learning Method via Parallel Tool Invocationcited by 16
  25. 2024Learning Evolving Tools for Large Language Modelscited by 15
  26. 2024Medchain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequencecited by 15
  27. 2025LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on Challenging Queriescited by 15
  28. 2025PLAY2PROMPT: Zero-shot Tool Instruction Optimization for LLM Agents via Tool Playcited by 14
  29. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  30. 2025LLM-Augmented Chemical Synthesis and Design Decision Programscited by 13
  31. 2025CTRLS: Chain-of-Thought Reasoning via Latent State-Transitioncited by 13
  32. 2026ToolGate: Contract-Grounded and Verified Tool Execution for LLMscited by 13
  33. 2025Better Process Supervision with Bi-directional Rewarding Signalscited by 12
  34. 2024BBox-Adapter: Lightweight Adapting for Black-Box Large Language Modelscited by 11
  35. 2024PreAct: Prediction Enhances Agent's Planning Abilitycited by 11
  36. 2024Budget-Constrained Tool Learning with Planningcited by 11
  37. 2024TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Usecited by 11
  38. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 11

50 most cited of 103 All 103 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Query-Efficient Planning with Language Models
  2. 2025Multi-Mission Tool Bench: Assessing the Robustness of LLM based Agents through Related and Dynamic Missions
  3. 2026Long-Horizon Plan Execution in Large Tool Spaces through Entropy-Guided Branching
  4. 2026PseudoAct: Leveraging Pseudocode Synthesis for Flexible Planning and Action Control in Large Language Model Agents
  5. 2026GATS: Graph-Augmented Tree Search with Layered World Models for Efficient Agent Planning
  6. 2026Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method
  7. 2026ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning
  8. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks