mlsanityData to 7 Oct 2026

Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Data

arXiv 2602.21320 ↗First posted 24 Feb 2026cs.LGcited by 31

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

also in 1 more
Abstract (arXiv)

Large language models (LLMs) are becoming the foundation for autonomous agents that can use tools to solve complex tasks. Reinforcement learning (RL) has emerged as a common approach for injecting such agentic capabilities, but typically under tightly controlled training setups. It often depends on carefully constructed task-solution pairs and substantial human supervision, which creates a fundamental obstacle to open-ended self-evolution toward superintelligent systems. In this paper, we propose Tool-R0 framework for training general purpose tool-calling agents from scratch with self-play RL, under a zero-data assumption. Initialized from the same base LLM, Tool-R0 co-evolves a Generator and a Solver with complementary rewards: one proposes targeted challenging tasks at the other's competence frontier and the other learns to solve them with real-world tool calls. This creates a self-evolving cycle that requires no pre-existing tasks or datasets. Evaluation on different tool-use benchmarks show that Tool-R0 yields 92.5 relative improvement over the base model and surpasses fully supervised tool-calling baselines under the same setting. Our work further provides empirical insights into self-play LLM agents by analyzing co-evolution, curriculum dynamics, and scaling behavior.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 46 of its 61 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

46 of 61 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey2 Sep
  2. 2025Agentic Reinforced Policy Optimization26 Jul
  3. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning30 Jun
  4. 2025ToolRL: Reward is All Tool Learning Needs16 Apr
  5. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMs15 Apr

Other references · 41

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  7. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  8. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  9. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  10. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  11. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  12. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
29 more
  1. 2022Large Language Models Can Self-Improvecited by 656
  2. 2024Self-Rewarding Language Modelscited by 654
  3. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  4. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  5. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 383
  6. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
  7. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  8. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  9. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  10. 2025TTRL: Test-Time Reinforcement Learningcited by 253
  11. 2024Tool Learning with Large Language Models: A Surveycited by 221
  12. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 210
  13. 2024ToolACE: Winning the Points of LLM Function Callingcited by 204
  14. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scalingcited by 110
  15. 2024xLAM: A Family of Large Action Models to Empower AI Agent Systemscited by 108
  16. 2024Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Modelscited by 97
  17. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 88
  18. 2024Self-playing Adversarial Language Game Enhances LLM Reasoningcited by 80
  19. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 78
  20. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 68
  21. 2024Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmarkcited by 62
  22. 2025Self-Challenging Language Model Agentscited by 56
  23. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 51
  24. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Datacited by 50
  25. 2025SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoningcited by 40
  26. 2026Dr. Zero: Self-Evolving Search Agents without Training Datacited by 39
  27. 2025Self-Improving LLM Agents at Test-Timecited by 30
  28. 2025Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoningcited by 24
  29. 2025Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Modelcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 31.

31 papers.

Sort:

By area:

  1. 2026PlanBench-XL: Evaluating Long-Horizon Planning of LLM Tool-Use Agents in Large-Scale Tool Ecosystemscited by 10
  2. 2026Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studiescited by 6
  3. 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Librariescited by 6
  4. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 6
  5. 2026Do Self-Evolving Agents Forget? Capability Degradation and Preservation in Lifelong LLM Agent Adaptationcited by 5
  6. 2026Evidence-RL: Towards Evidence-intensive Visual Reasoningcited by 3
  7. 2026PREPING: Building Agent Memory without Taskscited by 2
  8. 2026RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agentscited by 2
  9. 2026Why Does Feedback-Augmented Self-Distillation Fail to Improve Retrieval-Interleaved Search Agents?cited by 2
  10. 2026AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?cited by 2
  11. 2026What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agentscited by 2
  12. 2026AcquisitionSynthesis: Targeted Data Generation using Acquisition Functionscited by 1
19 more
  1. 2026Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoningcited by 1
  2. 2026Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learningcited by 1
  3. 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Designcited by 1
  4. 2026One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RLcited by 1
  5. 2026Self-Play Pretraining with Zero Datacited by 1
  6. 2026ANCORA: Learning to Question via Manifold-Anchored Self-Play for Verifiable Reasoningcited by 0
  7. 2026Confidence-Orchestrated Self-Evolution against Uncertain LLM Feedbackcited by 0
  8. 2026Eta Given Delta: Defining LLM Tool Efficiency With Marginal Tool Utilitycited by 0
  9. 2026From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvementcited by 0
  10. 2026SkillMentor: LLM Agent Self-Evolution via Learning Blind-Spot Diagnosiscited by 0
  11. 2026J-Zero: Unified Challenger--Solver--Judge Self-Evolution from Zero Datacited by 0
  12. 2026EvoUndo: Recoverability-Constrained Self-Evolution for LLM Agent Harnessescited by 0
  13. 2026SQL-Zero: Self-Evolving Text-to-SQLcited by 0
  14. 2026From Collaboration to Capability: Internalizing Routed LLM Experts into Compact Reasonerscited by 0
  15. 2026Self-Evolving Search Indexcited by 0
  16. 2026LensDesigner: A Self-Improving Agent for Optical Lens Designcited by 0
  17. 2026Self-Evolving Agents via Likelihood-Guided Tool-Space Optimizationcited by 0
  18. 2026When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gaincited by 0
  19. 2026Breaking Babel: A Self-Evolving Multi-Agent System for Long-Form Subtitle Translationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026From Self-Evolving Synthetic Data to Verifiable-Reward RL: Post-Training Multi-turn Interactive Tool-Using Agents
  2. 2026On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
  3. 2026Towards AGI A Pragmatic Approach Towards Self Evolving Agent
  4. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch
  5. 2025Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use
  6. 2024A Survey on Self-Evolution of Large Language Models
  7. 2025The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback
  8. 2025Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent