# APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay

arXiv: https://arxiv.org/abs/2504.03601 · HTML: https://mlsanity.com/paper/2504.03601/

## Key facts

- Cited by 167 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 42 of its 53 references that are in the map.
- First posted on arXiv 4 Apr 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 42)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [Large Language Models as Tool Makers](https://mlsanity.com/paper/2305.17126/) (2023)
- [Tool Learning with Foundation Models](https://mlsanity.com/paper/2304.08354/) (2023)
- [ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities](https://mlsanity.com/paper/2408.04682/) (2024)
- [InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback](https://mlsanity.com/paper/2306.14898/) (2023)
- [ToolACE: Winning the Points of LLM Function Calling](https://mlsanity.com/paper/2409.00920/) (2024)
- [APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets](https://mlsanity.com/paper/2406.18518/) (2024)
- [Agent S: An Open Agentic Framework that Uses Computers Like a Human](https://mlsanity.com/paper/2410.08164/) (2024)
- [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025)
- [SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement](https://mlsanity.com/paper/2410.20285/) (2024)
- [xLAM: A Family of Large Action Models to Empower AI Agent Systems](https://mlsanity.com/paper/2409.03215/) (2024)
- [Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning](https://mlsanity.com/paper/2412.09078/) (2024)
- [Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models](https://mlsanity.com/paper/2403.12881/) (2024)
- [Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments](https://mlsanity.com/paper/2501.10893/) (2025)
- [CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments](https://mlsanity.com/paper/2411.02305/) (2024)
- [Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent](https://mlsanity.com/paper/2411.02937/) (2024)
- [TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models](https://mlsanity.com/paper/2310.05905/) (2023)
- [LLM Agents Making Agent Tools](https://mlsanity.com/paper/2502.11705/) (2025)
- [Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning](https://mlsanity.com/paper/2410.12952/) (2024)
- [Magnet: Multi-turn Tool-use Data Synthesis and Distillation via Graph Translation](https://mlsanity.com/paper/2503.07826/) (2025)
- [Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents](https://mlsanity.com/paper/2408.07060/) (2024)

## Built on by (most cited first, 30 of 167)

- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model](https://mlsanity.com/paper/2508.14444/) (2025)
- [Agent Learning via Early Experience](https://mlsanity.com/paper/2510.08558/) (2025)
- [UserBench: An Interactive Gym Environment for User-Centric Agents](https://mlsanity.com/paper/2507.22034/) (2025)
- [Qwen3-Coder-Next Technical Report](https://mlsanity.com/paper/2603.00729/) (2026)
- [TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environments](https://mlsanity.com/paper/2510.01179/) (2025)
- [Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning](https://mlsanity.com/paper/2505.00024/) (2025)
- [Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning](https://mlsanity.com/paper/2602.10090/) (2026)
- [EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis](https://mlsanity.com/paper/2601.05808/) (2026)
- [Simulating Environments with Reasoning Models for Agent Training](https://mlsanity.com/paper/2511.01824/) (2025)
- [rStar2-Agent: Agentic Reasoning Technical Report](https://mlsanity.com/paper/2508.20722/) (2025)
- [Towards General Agentic Intelligence via Environment Scaling](https://mlsanity.com/paper/2509.13311/) (2025)
- [Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence](https://mlsanity.com/paper/2604.18292/) (2026)
- [CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions](https://mlsanity.com/paper/2505.18878/) (2025)
- [OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis](https://mlsanity.com/paper/2603.20278/) (2026)
- [ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training](https://mlsanity.com/paper/2602.06820/) (2026)
- [ToolMind Technical Report: A Large-Scale, Reasoning-Enhanced Tool-Use Dataset](https://mlsanity.com/paper/2511.15718/) (2025)
- [Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents](https://mlsanity.com/paper/2602.13379/) (2026)
- [Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents](https://mlsanity.com/paper/2605.22166/) (2026)
- [Goal Alignment in LLM-Based User Simulators for Conversational AI](https://mlsanity.com/paper/2507.20152/) (2025)
- [From Self-Evolving Synthetic Data to Verifiable-Reward RL: Post-Training Multi-turn Interactive Tool-Using Agents](https://mlsanity.com/paper/2601.22607/) (2026)
- [Enterprise Deep Research: Steerable Multi-Agent Deep Research for Enterprise Analytics](https://mlsanity.com/paper/2510.17797/) (2025)
- [Test-Time Adaptation for LLM Agents via Environment Interaction](https://mlsanity.com/paper/2511.04847/) (2025)
- [RealMem: Benchmarking LLMs in Real-World Memory-Driven Interaction](https://mlsanity.com/paper/2601.06966/) (2026)
- [Terminal-World: Scaling Terminal-Agent Environments via Agent Skills](https://mlsanity.com/paper/2605.20876/) (2026)
- [Don't Just Fine-tune the Agent, Tune the Environment](https://mlsanity.com/paper/2510.10197/) (2025)
- [Non-Collaborative User Simulators for Tool Agents](https://mlsanity.com/paper/2509.23124/) (2025)
- [ToolRM: Outcome Reward Models for Tool-Calling Large Language Models](https://mlsanity.com/paper/2509.11963/) (2025)
- [Generalizable End-to-End Tool-Use RL with Synthetic CodeGym](https://mlsanity.com/paper/2509.17325/) (2025)
- [Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch](https://mlsanity.com/paper/2511.01934/) (2025)

## Abstract (arXiv)

Training effective AI agents for multi-turn interactions requires high-quality data that captures realistic human-agent dynamics, yet such data is scarce and expensive to collect manually. We introduce APIGen-MT, a two-phase framework that generates verifiable and diverse multi-turn agent data. In the first phase, our agentic pipeline produces detailed task blueprints with ground-truth actions, leveraging a committee of LLM reviewers and iterative feedback loops. These blueprints are then transformed into complete interaction trajectories through simulated human-agent interplay. We train a family of models -- the xLAM-2-fc-r series with sizes ranging from 1B to 70B parameters. Our models outperform frontier models such as GPT-4o and Claude 3.5 on τ-bench and BFCL benchmarks, with the smaller models surpassing their larger counterparts, particularly in multi-turn settings, while maintaining superior consistency across multiple trials. Comprehensive experiments demonstrate that our verified blueprint-to-details approach yields high-quality training data, enabling the development of more reliable, efficient, and capable agents. We open-source 5K synthetic data trajectories and the trained xLAM-2-fc-r models to advance research in AI agents. Models at https://huggingface.co/collections/Salesforce/xlam-2-67ef5be12949d8dcdae354c4; Dataset at https://huggingface.co/datasets/Salesforce/APIGen-MT-5k and Website at https://apigen-mt.github.io