mlsanityData to 7 Oct 2026

Democratizing Tool Learning with Environments Fully Simulated by a Free 8B Language Model

arXiv 2604.17739 ↗First posted 20 Apr 2026cs.LG cs.CLcited by 1

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Reinforcement learning (RL) has become a prevalent paradigm for training tool calling agents, which typically requires online interactive environments. Existing approaches either rely on training data with ground truth annotations or require advanced proprietary language models (LMs) to synthesize environments that keep fixed once created. In this work, we propose TRUSTEE, a cost-friendly method for training tool calling agents with dynamic environments fully simulated by free open-source LMs that can be as small as 8B, including task generation, user simulation, tool simulation and trajectory evaluation, paired with an adaptive curriculum learning mechanism that controls task difficulty during training. Our empirical results show that TRUSTEE outperforms baselines which require extra external resources in most cases. These confirm that, with a sufficiently sophisticated design, even simulated environments with a local 8B LM as the backbone could set a strong baseline for tool learning. We hope our proposed paradigm could democratize tool learning and inspire future research on environment scaling with limited resources.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 34 of its 37 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

34 of 37 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning28 Dec
  2. 2025GTM: Simulating the World of Tools for AI Agents4 Dec
  3. 2025Simulating Environments with Reasoning Models for Agent Training3 Nov
  4. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch2 Nov
  5. 2025UserRL: Training Interactive User-Centric Agent via Reinforcement Learning24 Sep
  6. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey2 Sep
  7. 2025Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning25 Apr
  8. 2025ToolRL: Reward is All Tool Learning Needs16 Apr

Other references · 26

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  8. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  9. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  10. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  11. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  12. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
14 more
  1. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
  2. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 383
  3. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 273
  4. 2024ToolACE: Winning the Points of LLM Function Callingcited by 204
  5. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 203
  6. 2024StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Modelscited by 129
  7. 2024xLAM: A Family of Large Action Models to Empower AI Agent Systemscited by 108
  8. 2025Towards General Agentic Intelligence via Environment Scalingcited by 51
  9. 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learningcited by 51
  10. 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesiscited by 46
  11. 2024Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuningcited by 40
  12. 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?cited by 40
  13. 2025Procedural Environment Generation for Tool-Use Agentscited by 21
  14. 2025SynthTools: A Framework for Scaling Synthetic Tools for Agent Developmentcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22

Similar, not cited

Close in wording, no citation either way

  1. 2026Teaching LLMs to Learn Tool Trialing and Execution through Environment Interaction
  2. 2026On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
  3. 2025Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments
  4. 2026Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Data
  5. 2026Joint Optimization of Tool Creation and Use for Large Language Model Agents
  6. 2026SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Models
  7. 2025Adaptive Tool Generation with Models as Tools and Reinforcement Learning
  8. 2026Synthesize and Reward -- Reinforcement Learning for Multi-Step Tool Use in Live Environments