mlsanityData to 7 Oct 2026

ToolACE: Winning the Points of LLM Function Calling

arXiv 2409.00920 ↗First posted 2 Sep 2024cs.LG cs.AI cs.CLcited by 204

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Function calling significantly extends the application boundary of large language models, where high-quality and diverse training data is critical for unlocking this capability. However, real function-calling data is quite challenging to collect and annotate, while synthetic data generated by existing pipelines tends to lack coverage and accuracy. In this paper, we present ToolACE, an automatic agentic pipeline designed to generate accurate, complex, and diverse tool-learning data. ToolACE leverages a novel self-evolution synthesis process to curate a comprehensive API pool of 26,507 diverse APIs. Dialogs are further generated through the interplay among multiple agents, guided by a formalized thinking process. To ensure data accuracy, we implement a dual-layer verification system combining rule-based and model-based checks. We demonstrate that models trained on our synthesized data, even with only 8B parameters, achieve state-of-the-art performance on the Berkeley Function-Calling Leaderboard, rivaling the latest GPT-4 models. Our model and a subset of the data are publicly available at https://huggingface.co/Team-ACE.

Key facts. Cited by 204 papers in the map (data through 7 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 30 of its 37 references that are in the map. First posted on arXiv 2 Sep 2024.

Builds on

30 of 37 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks27 Jun
  2. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets26 Jun
  3. 2024Tool Learning with Large Language Models: A Survey28 May
  4. 2024API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs23 Feb
  5. 2024ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios1 Jan
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  7. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases8 Jun
  8. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
  9. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr

Other references · 21

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2023Qwen Technical Reportcited by 3,273
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  10. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  11. 2020Aligning AI With Shared Human Valuescited by 706
  12. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
9 more
  1. 2023MoDS: Model-oriented Data Selection for Instruction Tuningcited by 107
  2. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 88
  3. 2024Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarioscited by 85
  4. 2023Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Modelscited by 66
  5. 2024Comprehensive Exploration of Synthetic Data Generation: A Surveycited by 59
  6. 2024Large Language Models Can Solve Real-World Planning Rigorously with Formal Verification Toolscited by 47
  7. 2023LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automationcited by 40
  8. 2024Instruction Diversity Drives Generalization To Unseen Taskscited by 15
  9. 2024Equipping Language Models with Tool Use Capability for Tabular Data Analysis in Financecited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 88; 2026: 107.

204 papers.

Sort:

By area:

  1. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  2. 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 175
  3. 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 158
  4. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  5. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  6. 2025NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelcited by 94
  7. 2025Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scalingcited by 93
  8. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 88
  9. 2025ACEBench: Who Wins the Match Point in Tool Usage?cited by 66
  10. 2026Qwen3-Coder-Next Technical Reportcited by 60
  11. 2025TxAgent: An AI Agent for Therapeutic Reasoning Across a Universe of Toolscited by 57
  12. 2025G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systemscited by 56
38 more
  1. 2025ComplexFuncBench: Exploring Multi-Step and Constrained Function Calling under Long-Context Scenariocited by 53
  2. 2025Simulating Environments with Reasoning Models for Agent Trainingcited by 52
  3. 2025Towards General Agentic Intelligence via Environment Scalingcited by 51
  4. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  5. 2025ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Usecited by 48
  6. 2024From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactionscited by 46
  7. 2025Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for Large Language Modelscited by 46
  8. 2024NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Callscited by 45
  9. 2025ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptionscited by 44
  10. 2025Magnet: Multi-turn Tool-use Data Synthesis and Distillation via Graph Translationcited by 37
  11. 2025VideoAgent2: Enhancing the LLM-Based Agent System for Long-Form Video Understanding by Uncertainty-Aware CoTcited by 36
  12. 2026Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 33
  13. 2026Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Datacited by 31
  14. 2026ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosiscited by 30
  15. 2025Adaptive Tool Use in Large Language Models with Meta-Cognition Triggercited by 29
  16. 2026Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agentscited by 28
  17. 2024ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesiscited by 27
  18. 2025We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systemscited by 27
  19. 2025ToolMind Technical Report: A Large-Scale, Reasoning-Enhanced Tool-Use Datasetcited by 27
  20. 2024Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Basescited by 23
  21. 2026ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Trainingcited by 23
  22. 2025Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection Learningcited by 22
  23. 2025Procedural Environment Generation for Tool-Use Agentscited by 21
  24. 2025Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Modelcited by 20
  25. 2025SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RLcited by 20
  26. 2025AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Modelscited by 19
  27. 2025CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarioscited by 19
  28. 2025DrugPilot: LLM-based Parameterized Reasoning Agent for Drug Discoverycited by 18
  29. 2025Don't Just Fine-tune the Agent, Tune the Environmentcited by 17
  30. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratchcited by 17
  31. 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applicationscited by 17
  32. 2025Divide-Then-Aggregate: An Efficient Tool Learning Method via Parallel Tool Invocationcited by 16
  33. 2025AgentScope 1.0: A Developer-Centric Framework for Building Agentic Applicationscited by 16
  34. 2025ToolRM: Outcome Reward Models for Tool-Calling Large Language Modelscited by 16
  35. 2024Enhancing Function-Calling Capabilities in LLMs: Strategies for Prompt Formats, Data Integration, and Multilingual Translationcited by 14
  36. 2025PLAY2PROMPT: Zero-shot Tool Instruction Optimization for LLM Agents via Tool Playcited by 14
  37. 2025Towards Reliable Benchmarking: A Contamination Free, Controllable Evaluation Framework for Multi-step LLM Function Callingcited by 13
  38. 2026Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Textcited by 13

50 most cited of 204 All 204 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025FunReason-MT Technical Report: Advanced Data Synthesis Solution for Real-world Multi-Turn Tool-use
  2. 2025ToolFuzz -- Automated Agent Tool Testing
  3. 2026Try, Check and Retry: A Divide-and-Conquer Framework for Boosting Long-context Tool-Calling Performance of LLMs
  4. 2026ToolRobustBench: Stage-Wise Perturbation Evaluation and Failure Diagnosis for Tool-Calling Agents
  5. 2026Toolcompass: Guiding Tool Trialing, Not Suppressing It
  6. 2024TinyAgent: Function Calling at the Edge
  7. 2024Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agents
  8. 2026ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents