ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback
On the map: LLM agents and tool use → Tool learning and function calling
Abstract (arXiv)
High-quality tool-use data is critical for training language models to interact effectively with external tools. However, existing synthetic approaches typically follow a generate-then-filter paradigm with static post-hoc verification, often yielding inefficient data with imbalanced feature distributions. We propose ToolLoop, a closed-loop framework that decomposes synthesis into three progressive stages: (1) sampling function name combinations as ground truth; (2) backward derivation of user queries; and (3) forward derivation of tool calls. At each stage, dynamic self-feedback iteratively guides the model toward high-quality generation, realizing a transition from generate-then-filter to generate-verify-refine. On the Berkeley Function Calling Leaderboard (BFCL), a 4B parameter model trained with our 11K synthetic examples achieves 86.40% accuracy in non-reasoning mode, while an Isolate variant that removes BFCL-overlapping candidate functions still reaches 86.07%. Cross-benchmark evaluation on ACEBench further demonstrates strong generalization, with 72.1% overall accuracy using only 18.3% of baseline training data.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 19 of its 27 references that are in the map. First posted on arXiv 8 Sep 2026.
Builds on
19 of 27 references.
Closest ancestors · 3
Same direction, newest first
- 2025Enhancing LLM Tool Use with High-quality Instruction Data from Knowledge Graph26 Jun
- 2024ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesis24 Oct
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
Other references · 16
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
- 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
- 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
- 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 243
- 2024Agent S: An Open Agentic Framework that Uses Computers Like a Humancited by 187
- 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 167
- 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 149
- 2024SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinementcited by 132
- 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automationcited by 131
- 2024Evaluating Language Models as Synthetic Data Generatorscited by 33
- 2025ToolMind Technical Report: A Large-Scale, Reasoning-Enhanced Tool-Use Datasetcited by 30
4 more
- 2025An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliabilitycited by 19
- 2025YESciEval: Robust LLM-as-a-Judge for Scientific Question Answeringcited by 16
- 2025Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judgecited by 13
- 2025Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Modelscited by 8
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024ToolACE: Winning the Points of LLM Function Calling
- 2025CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios
- 2025Towards Reliable Benchmarking: A Contamination Free, Controllable Evaluation Framework for Multi-step LLM Function Calling
- 2026Toolcompass: Guiding Tool Trialing, Not Suppressing It
- 2025ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use
- 2026ToolWeave: Structured Synthesis of Complex Multi-Turn Tool-Calling Dialogues
- 2024TOOLVERIFIER: Generalization to New Tools via Self-Verification
- 2026Benchmarking the Benchmarks: A Validity Audit of Tool-Calling Evaluation