mlsanityData to 7 Oct 2026

Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmark

arXiv 2405.08355 ↗First posted 14 May 2024cs.CLcited by 62

On the map: LLM agents and tool use → Tool learning and function calling

also in 1 more
Abstract (arXiv)

This paper presents a new tool learning dataset Seal-Tools, which contains self-instruct API-like tools. Seal-Tools not only offers a large number of tools, but also includes instances which demonstrate the practical application of tools. Seeking to generate data on a large scale while ensuring reliability, we propose a self-instruct method to generate tools and instances, allowing precise control over the process. Moreover, our Seal-Tools contains hard instances that call multiple tools to complete the job, among which some are nested tool callings. For precise and comprehensive evaluation, we use strict format control and design three metrics from different dimensions. Therefore, Seal-Tools can serve as a new benchmark to evaluate the tool-calling ability of LLMs. Finally, we evaluate several prevalent LLMs and our finetuned model on Seal-Tools. The results show that current systems are far from perfect. The code, data and experiment results are available at https://github.com/fairyshine/Seal-Tools .

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 16 of its 28 references that are in the map. First posted on arXiv 14 May 2024.

Builds on

16 of 28 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  2. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases8 Jun
  3. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
  4. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings19 May
  5. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb

Other references · 10

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 426
  4. 2023In-Context Learning Creates Task Vectorscited by 302
  5. 2020MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselinescited by 201
  6. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  7. 2019Taskmaster-1: Toward a Realistic and Diverse Dialog Datasetcited by 139
  8. 2023GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Informationcited by 115
  9. 2023TaskWeaver: A Code-First Agent Frameworkcited by 85
  10. 2023Auto-ICL: In-Context Learning without Human Supervisioncited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 29; 2026: 22.

62 papers.

Sort:

By area:

  1. 2024Tool Learning with Large Language Models: A Surveycited by 221
  2. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  3. 2025Survey on Evaluation of LLM-based Agentscited by 190
  4. 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Modelcited by 101
  5. 2024Hammer: Robust Function-Calling for On-Device Language Models via Function Maskingcited by 53
  6. 2025ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Usecited by 48
  7. 2025LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?cited by 47
  8. 2024NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Callscited by 45
  9. 2024Benchmarking Agentic Workflow Generationcited by 36
  10. 2025ScaleMCP: Dynamic and Auto-Synchronizing Model Context Protocol Tools for LLM Agentscited by 36
  11. 2026Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Datacited by 31
  12. 2025Self-Improving LLM Agents at Test-Timecited by 30
38 more
  1. 2024Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Basescited by 23
  2. 2024Learning to Refuse: Towards Mitigating Privacy Risks in LLMscited by 22
  3. 2025MemTool: Optimizing Short-Term Memory Management for Dynamic Tool Calling in LLM Agent Multi-Turn Conversationscited by 21
  4. 2025ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filteringcited by 19
  5. 2026Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Taskscited by 18
  6. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratchcited by 17
  7. 2025ToolRM: Outcome Reward Models for Tool-Calling Large Language Modelscited by 16
  8. 2025Graph RAG-Tool Fusioncited by 15
  9. 2025LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on Challenging Queriescited by 15
  10. 2025Tool-to-Agent Retrieval: Bridging Tools and Agents for Scalable LLM Multi-Agent Systemscited by 15
  11. 2025StableToolBench-MirrorAPI: Modeling Tool Environments as Mirrors of 7,000+ Real-World APIscited by 14
  12. 2025CheMatAgent: Enhancing LLMs for Chemistry and Materials Science through Tree-Search Based Tool Learningcited by 12
  13. 2025GTM: Simulating the World of Tools for AI Agentscited by 12
  14. 2024TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Usecited by 11
  15. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 11
  16. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 11
  17. 2025A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directionscited by 9
  18. 2025MCP-Flow: Facilitating LLM Agents to Master Real-World, Diverse and Scaling MCP Toolscited by 9
  19. 2025AgentArch: A Comprehensive Benchmark to Evaluate Agent Architectures in Enterprisecited by 8
  20. 2025Enhancing Tool Learning in Large Language Models with Hierarchical Error Checklistscited by 7
  21. 2025Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Trainingcited by 6
  22. 2025FamilyTool: A Multi-hop Personalized Tool Use Benchmarkcited by 5
  23. 2024NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of Large Language Modelscited by 4
  24. 2026JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agentscited by 4
  25. 2025UAQFact: Evaluating Factual Knowledge Utilization of LLMs on Unanswerable Questionscited by 3
  26. 2026UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agentscited by 3
  27. 2026AsyncTool: Evaluating the Asynchronous Function Calling Capability under Multi-Task Scenarioscited by 3
  28. 2026Looking Is Not Picking: An Attention-Segment Account of Tool-Selection Failures in LLM Agentscited by 3
  29. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  30. 2026MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulationcited by 2
  31. 2026Skill2Query: Exploiting Skill Structure to Generate Pseudo-Queries for Agent Skill Retrievalcited by 2
  32. 2026Risk-Aware Reranking for Agentic Tool Retrievalcited by 2
  33. 2026What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agentscited by 2
  34. 2024Action Engine: Automatic Workflow Generation in FaaScited by 1
  35. 2024HammerBench: Fine-Grained Function-Calling Evaluation in Real Mobile Device Scenarioscited by 1
  36. 2025SELT: Self-Evaluation Tree Search for LLMs with Task Decompositioncited by 1
  37. 2025In-N-Out: A Parameter-Level API Graph Dataset for Tool Agentscited by 1
  38. 2026SimpleTool: Parallel Decoding for Real-Time LLM Function Callingcited by 1

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning to Rewrite Tool Descriptions for Reliable LLM-Agent Tool Use
  2. 2026SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?
  3. 2026OpaqueToolsBench: Learning Nuances of Tool Behavior Through Interaction
  4. 2026ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents
  5. 2026From Failure to Mastery: Generating Hard Samples for Tool-use Agents
  6. 2026ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents
  7. 2026DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Use
  8. 2026Toollery: Scaling LLM Agents to Thousands of Skills and Tools