mlsanityData to 7 Oct 2026

Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning

arXiv 2505.01441 ↗First posted 28 Apr 2025cs.AIcited by 107

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Large language models (LLMs) have achieved remarkable progress in complex reasoning tasks, yet they remain fundamentally limited by their reliance on static internal knowledge and text-only reasoning. Real-world problem solving often demands dynamic, multi-step reasoning, adaptive decision making, and the ability to interact with external tools and environments. In this work, we introduce ARTIST (Agentic Reasoning and Tool Integration in Self-improving Transformers), a unified framework that tightly couples agentic reasoning, reinforcement learning, and tool integration for LLMs. ARTIST enables models to autonomously decide when, how, and which tools to invoke within multi-turn reasoning chains, leveraging outcome-based RL to learn robust strategies for tool use and environment interaction without requiring step-level supervision. Extensive experiments on mathematical reasoning and multi-turn function calling benchmarks show that ARTIST consistently outperforms state-of-the-art baselines, with up to 22% absolute improvement over base models and strong gains on the most challenging tasks. Detailed studies and metric analyses reveal that agentic RL training leads to deeper reasoning, more effective tool use, and higher-quality solutions. Our results establish agentic RL with tool integration as a powerful new frontier for robust, interpretable, and generalizable problem-solving in LLMs.

Key facts. Cited by 107 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 27 of its 56 references that are in the map. First posted on arXiv 28 Apr 2025.

Builds on

27 of 56 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMs15 Apr
  2. 2025Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Tools7 Feb
  3. 2023ART: Automatic multi-step reasoning and tool-use for large language models16 Mar

Other references · 24

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  10. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  11. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  12. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
12 more
  1. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
  2. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 276
  3. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
  4. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learningcited by 152
  5. 2023ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Searchcited by 103
  6. 2024Building Math Agents with Multi-Turn Iterative Preference Learningcited by 66
  7. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approachescited by 43
  8. 2023MultiTool-CoT: GPT-3 Can Use Multiple External Tools with Chain of Thought Promptingcited by 27
  9. 2023Human-in-the-Loop through Chain-of-Thoughtcited by 20
  10. 2024Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neuronscited by 15
  11. 2024Beyond Lines and Circles: Unveiling the Geometric Reasoning Gap in Large Language Modelscited by 11
  12. 2025Self-Training Large Language Models for Tool-Use Without Demonstrationscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 48; 2026: 59.

107 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  2. 2025Agentic Reinforced Policy Optimizationcited by 166
  3. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  4. 2026MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memorycited by 113
  5. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 99
  6. 2025Reinforcing Multi-Turn Reasoning in LLM Agents via Fine-Grained Reward Structure and Credit Assignmentcited by 77
  7. 2025Tree Search for LLM Agent Reinforcement Learningcited by 73
  8. 2025Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learningcited by 67
  9. 2025Visual Agentic Reinforcement Fine-Tuningcited by 53
  10. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  11. 2025Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directionscited by 46
  12. 2025Distilling LLM Agent into Small Models with Retrieval and Code Toolscited by 41
38 more
  1. 2025Search and Refine During Think: Facilitating Knowledge Refinement for Improved Retrieval-Augmented Reasoningcited by 34
  2. 2025Demystifying Reinforcement Learning in Agentic Reasoningcited by 32
  3. 2025SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learningcited by 31
  4. 2025MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool usecited by 30
  5. 2025UserRL: Training Interactive User-Centric Agent via Reinforcement Learningcited by 27
  6. 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
  7. 2025PALADIN: Self-Correcting Language Model Agents to Cure Tool-Failure Casescited by 21
  8. 2025RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Usecited by 17
  9. 2025DeepTravel: An End-to-End Agentic Reinforcement Learning Framework for Autonomous Travel Planning Agentscited by 17
  10. 2025Don't Just Fine-tune the Agent, Tune the Environmentcited by 17
  11. 2025AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agentcited by 17
  12. 2026SOD: Step-wise On-policy Distillation for Small Language Model Agentscited by 17
  13. 2026Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuningcited by 15
  14. 2025WebCoT: Enhancing Web Agent Reasoning by Reconstructing Chain-of-Thought in Reflection, Branching, and Rollbackcited by 14
  15. 2025AutoGPS: Automated Geometry Problem Solving via Multimodal Formalization and Deductive Reasoningcited by 14
  16. 2025Generalizable End-to-End Tool-Use RL with Synthetic CodeGymcited by 14
  17. 2025Reinforcement Learning Foundations for Deep Research Systems: A Surveycited by 13
  18. 2025Stop-RAG: Value-Based Retrieval Control for Iterative RAGcited by 13
  19. 2025StepFun-Prover Preview: Let's Think and Verify Step by Stepcited by 12
  20. 2025GTM: Simulating the World of Tools for AI Agentscited by 12
  21. 2025AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoningcited by 11
  22. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 11
  23. 2026Synthetic Sandbox for Training Machine Learning Engineering Agentscited by 11
  24. 2025Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoningcited by 10
  25. 2025Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive Exploration for Agentic Reinforcement Learningcited by 10
  26. 2025Repurposing Synthetic Data for Fine-grained Search Agent Supervisioncited by 10
  27. 2026Over-Searching in Search-Augmented Large Language Modelscited by 10
  28. 2026NEWTON: Agentic Planning for Physically Grounded Video Generationcited by 10
  29. 2025THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoningcited by 9
  30. 2026Search-E1: Self-Distillation Drives Self-Evolution in Search-Augmented Reasoningcited by 9
  31. 2025Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AIcited by 8
  32. 2025CuES: A Curiosity-driven and Environment-grounded Synthesis Framework for Agentic RLcited by 8
  33. 2026ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognitioncited by 8
  34. 2026TRIP-Bench: A Benchmark for Long-Horizon Interactive Agents in Real-World Scenarioscited by 8
  35. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  36. 2026Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Usecited by 7
  37. 2026To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Callingcited by 7
  38. 2025SciAgent: A Unified Multi-Agent System for Generalistic Scientific Reasoningcited by 6

50 most cited of 107 All 107 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025AlphaApollo: A System for Deep Agentic Reasoning
  2. 2025Agent-as-Tool: A Study on the Hierarchical Decision Making with Reinforcement Learning
  3. 2026ToolVerse: Unlocking Massive Environments and Long-Horizon Tasks for Agentic Reinforcement Learning
  4. 2025AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learning
  5. 2025Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use
  6. 2026On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
  7. 2026Agentic Proposing: Enhancing Large Language Model Reasoning via Compositional Skill Synthesis
  8. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems