SynthTools: A Framework for Scaling Synthetic Tools for Agent Development
On the map: LLM agents and tool use → Tool learning and function calling
Abstract (arXiv)
For agentic systems to use external tools to solve complex, long-horizon tasks, we need a large set of diverse and controllable tool-use environments. We introduce SynthTools, a fully LLM-based pipeline spanning the entire lifecycle: environment generation, simulation, validation and task construction. By operating end-to-end through LLMs, our framework complements other tool-use environments bottlenecked by the complexity of real APIs, and ensures scalability and controllability by design. The framework consists of three components: top-down environment generation, which hierarchically constructs diverse, domain-grounded tool environments; environment simulation and validation, which ensures tools can be reliably emulated and filters out those that cannot; and bottom-up task and trajectory generation, which produces solvable and verifiable tasks together with multi-step trajectories, exposing control over difficulty, length, trajectory composition, and domain focus to guarantee flexibility. As a concrete instantiation, we release the dataset comprising 73,883 validated tools across 6,800 environments and 100 fields, 79,925 verifiable tasks as well as the pipeline to generate trajectories at scale. Training Qwen3 models of various sizes on a corpus of trajectories generated from these tasks yields gains across multiple tool-use benchmarks, including real APIs, indicating tool-use capabilities trained on synthetic data may transfer to some real environments. Together, these results suggest that SynthTools can serve as a useful infrastructure for large-scale training of tool-use agents.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 24 of its 47 references that are in the map. First posted on arXiv 11 Nov 2025.
Builds on
24 of 47 references.
Closest ancestors · 11
Same direction, newest first
- 2025LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on Challenging Queries21 Aug
- 2025LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?3 Aug
- 2025Procedural Environment Generation for Tool-Use Agents21 May
- 2025StableToolBench-MirrorAPI: Modeling Tool Environments as Mirrors of 7,000+ Real-World APIs26 Mar
- 2025ACEBench: Who Wins the Match Point in Tool Usage?22 Jan
- 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback19 Sep
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
- 2023ToolQA: A Dataset for LLM Question Answering with External Tools23 Jun
- 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases8 Jun
- 2023Gorilla: Large Language Model Connected with Massive APIs24 May
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr
Other references · 13
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
- 2023SemDeDup: Data-efficient learning at web-scale through semantic deduplicationcited by 303
- 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 167
- 2025Training Agents Inside of Scalable World Modelscited by 136
- 2023On the Tool Manipulation Capability of Open-source Large Language Modelscited by 122
- 2024GTA: A Benchmark for General Tool Agentscited by 77
- 2025Simulating Environments with Reasoning Models for Agent Trainingcited by 57
- 2025Towards General Agentic Intelligence via Environment Scalingcited by 54
- 2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agentscited by 46
- 2025Scaling Synthetic Task Generation for Agents via Explorationcited by 23
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Usecited by 13
- 2026ToolVerse: Unlocking Massive Environments and Long-Horizon Tasks for Agentic Reinforcement Learningcited by 3
- 2026What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agentscited by 3
- 2026Democratizing Tool Learning with Environments Fully Simulated by a Free 8B Language Modelcited by 1
- 2026RIFT-Bench: Dynamic Red-teaming For Agentic AI Systemscited by 0
- 2026What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents27 Aug 2026
- 2026ToolVerse: Unlocking Massive Environments and Long-Horizon Tasks for Agentic Reinforcement Learning17 Jul 2026
- 2026RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems22 Jun 2026
- 2026Democratizing Tool Learning with Environments Fully Simulated by a Free 8B Language Model20 Apr 2026
- 2026DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Use10 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL
- 2026C-World: A Computer Use Agent Environment Creator
- 2026The World Won't Stay Still: Programmable Evolution for Agent Benchmarks
- 2025AgentOrchestra: Orchestrating Multi-Agent Intelligence with the Tool-Environment-Agent(TEA) Protocol
- 2026MAS-Lab: A Specification-Driven Validation Framework for Reliable Multi-Agent Systems
- 2025Automating Benchmark Design
- 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis
- 2026ToolWeave: Structured Synthesis of Complex Multi-Turn Tool-Calling Dialogues