mlsanityData to 14 Oct 2026

Don't Just Fine-tune the Agent, Tune the Environment

arXiv 2510.10197 ↗First posted 11 Oct 2025cs.AIcited by 19

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Large Language Model (LLM) agents show great promise for complex, multi-turn tool-use tasks, but their development is often hampered by the extreme scarcity of high-quality training data. Supervised fine-tuning (SFT) on synthetic data leads to overfitting, whereas standard reinforcement learning (RL) struggles with a critical cold-start problem and training instability. To address these challenges, we introduce Environment Tuning, a novel training paradigm that enables agents to learn complex behaviors directly from problem instances without relying on pre-collected expert trajectories. Environment Tuning orchestrates this learning process through a structured curriculum, actionable environment augmentation that provides corrective feedback, and fine-grained progress rewards to ensure stable and efficient exploration. Using only 400 problem instances from Berkeley Function-Calling Leaderboard (BFCL) benchmark, our method not only achieves competitive in-distribution performance against strong baselines but also demonstrates superior out-of-distribution generalization, overcoming the performance collapse common to SFT-based approaches. Our work presents a paradigm shift from supervised fine-tuning on static trajectories to dynamic, environment-based exploration, paving the way for training more robust and data-efficient agents. The code is available at https://github.com/inclusionAI/AWorld-RL/tree/main/EnvTuning.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 38 of its 53 references that are in the map. First posted on arXiv 11 Oct 2025.

Builds on

38 of 53 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning2 Sep
  2. 2025Agentic Reinforced Policy Optimization26 Jul
  3. 2025Group-in-Group Policy Optimization for LLM Agent Training16 May
  4. 2025Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving12 May
  5. 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning28 Apr
  6. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning24 Apr
  7. 2025ToolRL: Reward is All Tool Learning Needs16 Apr
  8. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMs15 Apr
  9. 2025ToRL: Scaling Tool-Integrated RL30 Mar

Other references · 29

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  7. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  8. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  9. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  10. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  11. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  12. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
17 more
  1. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  2. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  3. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  4. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  5. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 286
  6. 2024ToolACE: Winning the Points of LLM Function Callingcited by 216
  7. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 194
  8. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 176
  9. 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 167
  10. 2025Skywork Open Reasoner 1 Technical Reportcited by 165
  11. 2025OpenCUA: Open Foundations for Computer-Use Agentscited by 160
  12. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scalingcited by 113
  13. 2024xLAM: A Family of Large Action Models to Empower AI Agent Systemscited by 111
  14. 2025ACEBench: Who Wins the Match Point in Tool Usage?cited by 71
  15. 2025Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Modelscited by 70
  16. 2025Magnet: Multi-turn Tool-use Data Synthesis and Distillation via Graph Translationcited by 37
  17. 2025Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuningcited by 34

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 17.

19 papers.

Sort:

By area:

  1. 2026RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL Systemcited by 24
  2. 2026EnvHarness: Awakening Static Worlds for Agent Learningcited by 13
  3. 2026SEAL: Synergistic Co-Evolution of Agents and Learning Environmentscited by 12
  4. 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 12
  5. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 12
  6. 2026Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipecited by 8
  7. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  8. 2026Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix Itcited by 7
  9. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 5
  10. 2025CP-Env: Evaluating Large Language Models on Clinical Pathways in a Controllable Hospital Environmentcited by 4
  11. 2026Teach the Magnitude, Not the Direction: Verifier-Bounded Credit Assignment for Multi-Turn Multi-step LLM Agentscited by 4
  12. 2026Towards Agentic Intelligence for Materials Sciencecited by 3
7 more
  1. 2026Pushing the Limits of LLM Tool Calling via Experiential Knowledge Integration and Activationcited by 3
  2. 2026RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agentscited by 2
  3. 2025Assertion-Conditioned Compliance: A Provenance-Aware Vulnerability in Multi-Turn Tool-Calling Agentscited by 1
  4. 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Designcited by 1
  5. 2026UnifiedPlayers: Enhance Tool-Integrated Reasoning in Agentic Reinforcement Learningcited by 0
  6. 2026Beyond Instruction Following: Learning Grounded Skill-Following with Skill Contractscited by 0
  7. 2026Where to Start, How to Move: Analyzing Curricula under GRPO for Long-Horizon LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks
  2. 2026Can RL Improve Generalization of LLM Agents? An Empirical Study
  3. 2026State2State: Environment-Derived Mid-Training for LLM Agents
  4. 2025ATLaS: Agent Tuning via Learning Critical Steps
  5. 2023Shattering the Agent-Environment Interface for Fine-Tuning Inclusive Language Models
  6. 2026Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use
  7. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Models
  8. 2026Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL