mlsanityData to 14 Oct 2026

Procedural Environment Generation for Tool-Use Agents

arXiv 2506.11045 ↗First posted 21 May 2025cs.LGcited by 24

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Although the power of LLM tool-use agents has ignited a flurry of recent research in this area, the curation of tool-use training data remains an open problem-especially for online RL training. Existing approaches to synthetic tool-use data generation tend to be non-interactive, and/or non-compositional. We introduce RandomWorld, a pipeline for the procedural generation of interactive tools and compositional tool-use data. We show that models tuned via SFT and RL on synthetic RandomWorld data improve on a range of tool-use benchmarks, and set the new SoTA for two metrics on the NESTFUL dataset. Further experiments show that downstream performance scales with the amount of RandomWorld-generated training data, opening up the possibility of further improvement through the use of entirely synthetic data.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 24 of its 41 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

24 of 41 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls4 Sep
  2. 2024ToolACE: Winning the Points of LLM Function Calling2 Sep
  3. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets26 Jun
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  5. 2023ToolQA: A Dataset for LLM Question Answering with External Tools23 Jun
  6. 2023Gorilla: Large Language Model Connected with Massive APIs24 May

Other references · 18

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  5. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  6. 2023Large Language Modelscited by 780
  7. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  8. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
  9. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
  10. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 392
  11. 2025ToolRL: Reward is All Tool Learning Needscited by 362
  12. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 256
6 more
  1. 2024xLAM: A Family of Large Action Models to Empower AI Agent Systemscited by 111
  2. 2024Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarioscited by 86
  3. 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generationcited by 83
  4. 2024Large Language Models Empowered Personalized Web Agentscited by 66
  5. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 45
  6. 2024OpenResearcher: Unleashing AI for Accelerated Scientific Researchcited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 21.

24 papers.

Sort:

By area:

  1. 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learningcited by 66
  2. 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesiscited by 63
  3. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 54
  4. 2026EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RLcited by 14
  5. 2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
  6. 2026PlanBench-XL: Evaluating Long-Horizon Planning of LLM Tool-Use Agents in Large-Scale Tool Ecosystemscited by 11
  7. 2025GRPO is Secretly a Process Reward Modelcited by 10
  8. 2026ClawEnvKit: Automatic Environment Generation for Claw-Like Agentscited by 8
  9. 2025SynthTools: A Framework for Scaling Synthetic Tools for Agent Developmentcited by 5
  10. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 5
  11. 2026TopoCurate:Modeling Interaction Topology for Tool-Use Agent Trainingcited by 4
  12. 2026JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agentscited by 4
12 more
  1. 2026TRACE: Capability-Targeted Agentic Trainingcited by 3
  2. 2026Learning to Search: A Decision-Based Agent for Knowledge-Based Visual Question Answeringcited by 2
  3. 2026Scalable Environments Drive Generalizable Agentscited by 2
  4. 2026Democratizing Tool Learning with Environments Fully Simulated by a Free 8B Language Modelcited by 1
  5. 2026Accelerating Constrained Decoding with Token Space Compressioncited by 0
  6. 2026Failure-Guided Co-Evolution of Prompts and Training Datacited by 0
  7. 2026Toolcompass: Guiding Tool Trialing, Not Suppressing Itcited by 0
  8. 2026When Does Action Credit Need Updating?cited by 0
  9. 2026WEFT: Scaling Tool-Use Post-Training for General-Purpose Agentscited by 0
  10. 2026ThunderSyncRL: Lossless Acceleration of Agentic Reinforcement Learningcited by 0
  11. 2026Terrarium: Growing Personal Assistants via Scalable Synthesis of App Worldscited by 0
  12. 2026One Task, Many Interfaces: Simple Cross-Interface Consistency Training Improves LLM Agent Robustnesscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ToolWeave: Structured Synthesis of Complex Multi-Turn Tool-Calling Dialogues
  2. 2026SynAE: A Framework for Measuring the Quality of Synthetic Data for Tool-Calling Agent Evaluations
  3. 2026OpaqueToolsBench: Learning Nuances of Tool Behavior Through Interaction
  4. 2026E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios
  5. 2026Execution-First Synthetic Tool-Use Trace Generation for LLM Agents
  6. 2026DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Use
  7. 2026SKT: Skill-Use Training at Scale via Verified Synthetic Data Generation
  8. 2026From Self-Evolving Synthetic Data to Verifiable-Reward RL: Post-Training Multi-turn Interactive Tool-Using Agents