# AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents

arXiv: https://arxiv.org/abs/2506.14205 · HTML: https://mlsanity.com/paper/2506.14205/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 17 Jun 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 26 of 26)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?](https://mlsanity.com/paper/2403.07718/) (2024)
- [TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks](https://mlsanity.com/paper/2412.14161/) (2024)
- [Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration](https://mlsanity.com/paper/1802.08802/) (2018)
- [An Illusion of Progress? Assessing the Current State of Web Agents](https://mlsanity.com/paper/2504.01382/) (2025)
- [OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis](https://mlsanity.com/paper/2412.19723/) (2024)
- [Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents](https://mlsanity.com/paper/2504.00906/) (2025)
- [AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials](https://mlsanity.com/paper/2412.09605/) (2024)
- [AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories](https://mlsanity.com/paper/2504.08942/) (2025)
- [Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments](https://mlsanity.com/paper/2501.10893/) (2025)
- [WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks](https://mlsanity.com/paper/2407.05291/) (2024)
- [NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions](https://mlsanity.com/paper/2502.13124/) (2025)
- [Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agents](https://mlsanity.com/paper/2502.11357/) (2025)
- [NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wild](https://mlsanity.com/paper/2410.02907/) (2024)
- [AutoBencher: Towards Declarative Benchmark Construction](https://mlsanity.com/paper/2407.08351/) (2024)
- [Go-Browse: Training Web Agents with Structured Exploration](https://mlsanity.com/paper/2506.03533/) (2025)
- [Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web](https://mlsanity.com/paper/2311.18751/) (2023)
- [Synthetic Datasets for Neural Program Synthesis](https://mlsanity.com/paper/1912.12345/) (2019)

## Built on by (most cited first, 30 of 46)

- [Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning](https://mlsanity.com/paper/2602.10090/) (2026)
- [OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflows](https://mlsanity.com/paper/2508.09124/) (2025)
- [Agentic Web: Weaving the Next Web with AI Agents](https://mlsanity.com/paper/2507.21206/) (2025)
- [Scaling Agent Learning via Experience Synthesis](https://mlsanity.com/paper/2511.03773/) (2025)
- [InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training](https://mlsanity.com/paper/2601.04126/) (2026)
- [Scaling Synthetic Task Generation for Agents via Exploration](https://mlsanity.com/paper/2509.25047/) (2025)
- [WebGym: Scaling Training Environments for Visual Web Agents with Realistic Tasks](https://mlsanity.com/paper/2601.02439/) (2026)
- [Proactive Agent Research Environment: Simulating Active Users to Evaluate Proactive Assistants](https://mlsanity.com/paper/2604.00842/) (2026)
- [GUI-ReWalk: Massive Data Generation for GUI Agent via Stochastic Exploration and Intent-Aware Reasoning](https://mlsanity.com/paper/2509.15738/) (2025)
- [OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesis](https://mlsanity.com/paper/2604.15093/) (2026)
- [DatasetResearch: Benchmarking Agent Systems for Demand-Driven Dataset Discovery](https://mlsanity.com/paper/2508.06960/) (2025)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL](https://mlsanity.com/paper/2607.11185/) (2026)
- [NexForge: Scaling Agent Capabilities through Requirement-Driven Task Synthesis for LLMs](https://mlsanity.com/paper/2607.14186/) (2026)
- [OpenForgeRL: Train Harness-native Agents in Any Environment](https://mlsanity.com/paper/2607.21557/) (2026)
- [Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain](https://mlsanity.com/paper/2510.05159/) (2025)
- [SynthTools: A Framework for Scaling Synthetic Tools for Agent Development](https://mlsanity.com/paper/2511.09572/) (2025)
- [The World Won't Stay Still: Programmable Evolution for Agent Benchmarks](https://mlsanity.com/paper/2603.05910/) (2026)
- [Anchor: Mitigating Artifact Drift in Agent Benchmark Generation](https://mlsanity.com/paper/2605.26321/) (2026)
- [Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence](https://mlsanity.com/paper/2608.31075/) (2026)
- [Learning with Challenges: Adaptive Difficulty-Aware Data Generation for Mobile GUI Agent Training](https://mlsanity.com/paper/2601.22781/) (2026)
- [Agentic Reward Modeling: Verifying GUI Agent via Progressive Trajectory-Grounded Interaction](https://mlsanity.com/paper/2602.00575/) (2026)
- [Matrix: Peer-to-Peer Multi-Agent Synthetic Data Generation Framework](https://mlsanity.com/paper/2511.21686/) (2025)
- [Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning](https://mlsanity.com/paper/2606.27483/) (2026)
- [What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents](https://mlsanity.com/paper/2608.27260/) (2026)
- [Learning from Online Videos at Inference Time for Computer-Use Agents](https://mlsanity.com/paper/2511.04137/) (2025)
- [Scalable Environments Drive Generalizable Agents](https://mlsanity.com/paper/2605.18181/) (2026)
- [ProCUA-SFT Technical Report](https://mlsanity.com/paper/2606.17321/) (2026)
- [Execution-First Synthetic Tool-Use Trace Generation for LLM Agents](https://mlsanity.com/paper/2607.29175/) (2026)
- [OmnilingualGAIA2: Evaluating the Multilingual Gap in Frontier AI Agents](https://mlsanity.com/paper/2608.08775/) (2026)

## Abstract (arXiv)

We introduce AgentSynth, a scalable and cost-efficient pipeline for automatically synthesizing high-quality tasks and trajectory datasets for generalist computer-use agents. Leveraging information asymmetry, AgentSynth constructs subtasks that are simple during generation but significantly more challenging when composed into long-horizon tasks, enabling the creation of over 6,000 diverse and realistic tasks. A key strength of AgentSynth is its ability to precisely modulate task complexity by varying the number of subtasks. Empirical evaluations show that state-of-the-art LLM agents suffer a steep performance drop, from 18% success at difficulty level 1 to just 4% at level 6, highlighting the benchmark's difficulty and discriminative power. Moreover, our pipeline achieves a low average cost of 0.60 per trajectory, orders of magnitude cheaper than human annotations. Our code and data are available at https://github.com/sunblaze-ucb/AgentSynth