# Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents

arXiv: https://arxiv.org/abs/2601.20144 · HTML: https://mlsanity.com/paper/2601.20144/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use.
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 28 Jan 2026.

## Placed in

- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 20 of 20)

- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct](https://mlsanity.com/paper/2308.09583/) (2023)
- [AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents](https://mlsanity.com/paper/2407.18901/) (2024)
- [ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases](https://mlsanity.com/paper/2306.05301/) (2023)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay](https://mlsanity.com/paper/2504.03601/) (2025)
- [Asking Clarifying Questions in Open-Domain Information-Seeking Conversations](https://mlsanity.com/paper/1907.06554/) (2019)
- [MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers](https://mlsanity.com/paper/2508.14704/) (2025)
- [Recent Advances and Challenges in Task-oriented Dialog System](https://mlsanity.com/paper/2003.07490/) (2020)
- [UserBench: An Interactive Gym Environment for User-Centric Agents](https://mlsanity.com/paper/2507.22034/) (2025)
- [LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?](https://mlsanity.com/paper/2508.01780/) (2025)
- [Customer-R1: Personalized Simulation of Human Behaviors via RL-based LLM Agent in Online Shopping](https://mlsanity.com/paper/2510.07230/) (2025)
- [Non-Collaborative User Simulators for Tool Agents](https://mlsanity.com/paper/2509.23124/) (2025)

## Built on by (most cited first, 14 of 14)

- [Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning](https://mlsanity.com/paper/2605.02913/) (2026)
- [Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails](https://mlsanity.com/paper/2609.09134/) (2026)
- [SENTINEL: Failure-Driven Reinforcement Learning for Training Tool-Using Language Model Agents](https://mlsanity.com/paper/2606.12908/) (2026)
- [What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents](https://mlsanity.com/paper/2608.27260/) (2026)
- [WRIT: Write-Read Intensive Trajectory Synthesis for Multi-Turn User-Facing Agents](https://mlsanity.com/paper/2606.02908/) (2026)
- [Simulate to Generalize: Scaling Stateful Supervision for API-calling Agents using LLM World Models](https://mlsanity.com/paper/2607.16900/) (2026)
- [Retry, Switch, or Abstain? Learning Strategy-Aware Tool-Use Policies via Controlled Error Injection](https://mlsanity.com/paper/2608.11977/) (2026)
- [DRIP-R: A Benchmark for Decision-Making and Reasoning Under Real-World Policy Ambiguity in the Retail Domain](https://mlsanity.com/paper/2605.07699/) (2026)
- [Reinforcement Learning for Tool-Calling Agents in Fast Healthcare Interoperability Resources (FHIR)](https://mlsanity.com/paper/2605.14126/) (2026)
- [A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks](https://mlsanity.com/paper/2605.28556/) (2026)
- [ForestBench: A Unified Graph Framework for Evaluating Multi-Agent Collaboration](https://mlsanity.com/paper/2608.08605/) (2026)
- [D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transform](https://mlsanity.com/paper/2610.04938/) (2026)
- [Know What You Don't Know: Agentic Data Synthesis by Evolving Generation Space for Closed-Loop Learning](https://mlsanity.com/paper/2610.12593/) (2026)
- [Davinci-Tool: Scaling Agentic Tool-Use Supervision with Verifiable Executable Environments](https://mlsanity.com/paper/2610.16867/) (2026)

## Abstract (arXiv)

Tool-calling agents are increasingly deployed in real-world customer-facing workflows. Yet most studies on tool-calling agents focus on idealized settings with general, fixed, and well-specified tasks. In real-world applications, user requests are often (1) ambiguous, (2) changing over time, or (3) infeasible due to policy constraints, and training and evaluation data that cover these diverse, complex interaction patterns remain under-represented. To bridge the gap, we present Trajectory2Task, a verifiable data generation pipeline for studying tool use at scale under three realistic user scenarios: ambiguous intent, changing intent, and infeasible intents. The pipeline first conducts multi-turn exploration to produce valid tool-call trajectories. It then converts these trajectories into user-facing tasks with controlled intent adaptations. This process yields verifiable task that support closed-loop evaluation and training. We benchmark seven state-of-the-art LLMs on the generated complex user scenario tasks and observe frequent failures. Finally, using successful trajectories obtained from task rollouts, we fine-tune lightweight LLMs and find consistent improvements across all three conditions, along with better generalization to unseen tool-use domains, indicating stronger tool-calling ability.