mlsanityData to 7 Oct 2026

TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage

arXiv 2308.03427 ↗First posted 7 Aug 2023cs.AIcited by 54

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

With recent advancements in natural language processing, Large Language Models (LLMs) have emerged as powerful tools for various real-world applications. Despite their prowess, the intrinsic generative abilities of LLMs may prove insufficient for handling complex tasks which necessitate a combination of task planning and the usage of external tools. In this paper, we first propose a structured framework tailored for LLM-based AI Agents and discuss the crucial capabilities necessary for tackling intricate problems. Within this framework, we design two distinct types of agents (i.e., one-step agent and sequential agent) to execute the inference process. Subsequently, we instantiate the framework using various LLMs and evaluate their Task Planning and Tool Usage (TPTU) abilities on typical tasks. By highlighting key findings and challenges, our goal is to provide a helpful resource for researchers and practitioners to leverage the power of LLMs in their AI applications. Our study emphasizes the substantial potential of these models, while also identifying areas that need more investigation and improvement.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 80 of its 95 references that are in the map. First posted on arXiv 7 Aug 2023.

Builds on

80 of 95 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022A Generalist Agent12 May

Other references · 79

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  9. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  12. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
38 more
  1. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2023A Survey of Large Language Modelscited by 2,537
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  11. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  12. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  13. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  14. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  15. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  16. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  17. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  18. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  19. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  20. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  21. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  22. 2021GPT Understands, Toocited by 715
  23. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  24. 2022PAL: Program-aided Language Modelscited by 635
  25. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  26. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  27. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  28. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  29. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  30. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  31. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
  32. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  33. 2023Augmented Language Models: a Surveycited by 398
  34. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 383
  35. 2023Language Models can Solve Computer Taskscited by 371
  36. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  37. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  38. 2023Tool Learning with Foundation Modelscited by 302

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 19; 2025: 19; 2026: 7.

54 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  3. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  4. 2025Memory in the Age of AI Agentscited by 285
  5. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
  6. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  7. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  8. 2024Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarioscited by 85
  9. 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilitiescited by 82
  10. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  11. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  12. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
38 more
  1. 2023Experiential Co-Learning of Software-Developing Agentscited by 57
  2. 2024VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMscited by 47
  3. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  4. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  5. 2024A Survey of Useful LLM Evaluationcited by 28
  6. 2024ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesiscited by 27
  7. 2024Security of AI Agentscited by 24
  8. 2024TimeArena: Shaping Efficient Multitasking Language Agents in a Time-Aware Simulationcited by 22
  9. 2024A Trembling House of Cards? Mapping Adversarial Attacks against Language Agentscited by 22
  10. 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applicationscited by 17
  11. 2024Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Executioncited by 16
  12. 2025Encouraging Good Processes Without the Need for Good Answers: Reinforcement Learning for LLM Agent Planningcited by 15
  13. 2024VURF: A General-purpose Reasoning and Self-refinement Framework for Video Understandingcited by 13
  14. 2024MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generationcited by 12
  15. 2023Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulationscited by 11
  16. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 11
  17. 2024From Interaction to Impact: Towards Safer AI Agents Through Understanding and Evaluating Mobile UI Operation Impactscited by 10
  18. 2024MeNTi: Bridging Medical Calculator and LLM Agent with Nested Tool Callingcited by 10
  19. 2025HiMA-Ecom: Enabling Joint Training of Hierarchical Multi-Agent E-commerce Assistantscited by 9
  20. 2026The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systemscited by 8
  21. 2025ToolACE-DEV: Self-Improving Tool Learning via Decomposition and EVolutioncited by 7
  22. 2025Talk to Your Slides: High-Efficiency Slide Editing via Language-Driven Structured Data Manipulationcited by 7
  23. 2023A New Dialogue Response Generation Agent for Large Language Models by Asking Questions to Detect User's Intentionscited by 6
  24. 2023Secure Transformer Inference Protocolcited by 5
  25. 2025Position: Stop Acting Like Language Model Agents Are Normal Agentscited by 5
  26. 2025GasAgent: A Multi-Agent Framework for Automated Gas Optimization in Smart Contractscited by 5
  27. 2024Deploying Foundation Model Powered Agent Services: A Surveycited by 4
  28. 2025LLMTM: Benchmarking and Optimizing LLMs for Temporal Motif Analysis in Dynamic Graphscited by 4
  29. 2025ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoningcited by 3
  30. 2025AgenticIE: An Adaptive Agent for Information Extraction from Complex Regulatory Documentscited by 3
  31. 2024SEAL: Suite for Evaluating API-use of LLMscited by 2
  32. 2025Can Competition Enhance the Proficiency of Agents Powered by Large Language Models in the Realm of News-driven Time Series Forecasting?cited by 2
  33. 2025The Athenian Academy: A Seven-Layer Architecture Model for Multi-Agent Systemscited by 2
  34. 2025STARQA: A Question Answering Dataset for Complex Analytical Reasoning over Structured Databasescited by 2
  35. 2025PersonaAgent with GraphRAG: Community-Aware Knowledge Graphs for Personalized LLMcited by 2
  36. 2026Automated Post-Incident Policy Gap Analysis via Threat-Informed Evidence Mapping using Large Language Modelscited by 1
  37. 2026Hierarchical Alignment: Enforcing Hierarchical Instruction-Following in LLMs through Logical Consistencycited by 1
  38. 2023Learning Top-k Subtask Planning Tree based on Discriminative Representation Pre-training for Decision Makingcited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems
  2. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  3. 2023An In-depth Survey of Large Language Model-based Artificial Intelligence Agents
  4. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
  5. 2024Understanding the planning of LLM agents: A survey
  6. 2024Large Multimodal Agents: A Survey
  7. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges
  8. 2024Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights