# Scaling Synthetic Task Generation for Agents via Exploration

arXiv: https://arxiv.org/abs/2509.25047 · HTML: https://mlsanity.com/paper/2509.25047/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 18 of 18)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge](https://mlsanity.com/paper/2206.08853/) (2022)
- [Seed1.5-VL Technical Report](https://mlsanity.com/paper/2505.07062/) (2025)
- [Agent S: An Open Agentic Framework that Uses Computers Like a Human](https://mlsanity.com/paper/2410.08164/) (2024)
- [Tree Search for Language Model Agents](https://mlsanity.com/paper/2407.01476/) (2024)
- [GTA1: GUI Test-time Scaling Agent](https://mlsanity.com/paper/2507.05791/) (2025)
- [Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agents](https://mlsanity.com/paper/2412.13194/) (2024)
- [AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents](https://mlsanity.com/paper/2506.14205/) (2025)

## Built on by (most cited first, 23 of 23)

- [SkillX: Automatically Constructing Skill Knowledge Bases for Agents](https://mlsanity.com/paper/2604.04804/) (2026)
- [GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators](https://mlsanity.com/paper/2512.19682/) (2025)
- [UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action](https://mlsanity.com/paper/2510.17790/) (2025)
- [CoEvolve: Training LLM Agents via Agent-Data Mutual Evolution](https://mlsanity.com/paper/2604.15840/) (2026)
- [CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents](https://mlsanity.com/paper/2605.25624/) (2026)
- [Ferret-UI Lite: Lessons from Building Small On-Device GUI Agents](https://mlsanity.com/paper/2509.26539/) (2025)
- [SPADE: Self-Play in Adaptive Synthetic Executable Environments](https://mlsanity.com/paper/2608.19197/) (2026)
- [Safe and Scalable Web Agent Learning via Recreated Websites](https://mlsanity.com/paper/2603.10505/) (2026)
- [OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesis](https://mlsanity.com/paper/2604.15093/) (2026)
- [Expanding LLM Agent Boundaries with Strategy-Guided Exploration](https://mlsanity.com/paper/2603.02045/) (2026)
- [Learning to Build the Environment: Self-Evolving Reasoning RL via Verifiable Environment Synthesis](https://mlsanity.com/paper/2605.14392/) (2026)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [SynthTools: A Framework for Scaling Synthetic Tools for Agent Development](https://mlsanity.com/paper/2511.09572/) (2025)
- [UI-Oceanus: Scaling GUI Agents with Synthetic Environmental Dynamics](https://mlsanity.com/paper/2604.02345/) (2026)
- [ANCHOR: Branch-Point Data Generation for GUI Agents](https://mlsanity.com/paper/2602.07153/) (2026)
- [GUI Agents with Reinforcement Learning: Toward Digital Inhabitants](https://mlsanity.com/paper/2604.27955/) (2026)
- [What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents](https://mlsanity.com/paper/2608.27260/) (2026)
- [EnterpriseLab: A Full-Stack Platform for developing and deploying agents in Enterprises](https://mlsanity.com/paper/2603.21630/) (2026)
- [SynCo: Data Synthesis Co-Training for Self-Evolving LLMs via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2610.11345/) (2026)
- [Scaling Verifiable Environments for Long-horizon Work Agents](https://mlsanity.com/paper/2610.04906/) (2026)
- [Know What You Don't Know: Agentic Data Synthesis by Evolving Generation Space for Closed-Loop Learning](https://mlsanity.com/paper/2610.12593/) (2026)
- [Davinci-Tool: Scaling Agentic Tool-Use Supervision with Verifiable Executable Environments](https://mlsanity.com/paper/2610.16867/) (2026)
- [Explorer-Zero: Improving Self-Play for Long-Horizon Tasks with Decomposable Verifiable Rewards](https://mlsanity.com/paper/2610.19868/) (2026)

## Abstract (arXiv)

Post-Training Multimodal Large Language Models (MLLMs) to build interactive agents holds promise across domains such as computer-use, web navigation, and robotics. A key challenge in scaling such post-training is lack of high-quality downstream agentic task datasets with tasks that are diverse, feasible, and verifiable. Existing approaches for task generation rely heavily on human annotation or prompting MLLM with limited downstream environment information, which is either costly or poorly scalable as it yield tasks with limited coverage. To remedy this, we present AutoPlay, a scalable pipeline for task generation that explicitly explores interactive environments to discover possible interactions and current state information to synthesize environment-grounded tasks. AutoPlay operates in two stages: (i) an exploration phase, where an MLLM explorer agent systematically uncovers novel environment states and functionalities, and (ii) a task generation phase, where a task generator leverages exploration trajectories and a set of task guideline prompts as context to synthesize diverse, executable, and verifiable tasks. We show AutoPlay generates 20k tasks across 20 Android applications and 10k tasks across 13 applications Ubuntu applications to train mobile-use and computer-use agents. AutoPlay generated tasks enable large-scale task demonstration synthesis without human annotation by employing an MLLM task executor and verifier. This data enables training MLLM-based UI agents that improve success rates up to 20.0% on mobile-use and 10.9% on computer-use scenarios. In addition, AutoPlay generated tasks combined with MLLM verifier-based rewards enable scaling reinforcement learning training of UI agents, leading to an additional 5.7% gain. coverage. These results establish AutoPlay as a scalable approach for post-training capable MLLM agents reducing reliance on human annotation.