mlsanityData to 14 Oct 2026

AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learning

arXiv 2507.21836 ↗First posted 29 Jul 2025cs.CLcited by 33

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Large Language Models (LLMs), when enhanced through reasoning-oriented post-training, evolve into powerful Large Reasoning Models (LRMs). Tool-Integrated Reasoning (TIR) further extends their capabilities by incorporating external tools, but existing methods often rely on rigid, predefined tool-use patterns that risk degrading core language competence. Inspired by the human ability to adaptively select tools, we introduce AutoTIR, a reinforcement learning framework that enables LLMs to autonomously decide whether and which tool to invoke during the reasoning process, rather than following static tool-use strategies. AutoTIR leverages a hybrid reward mechanism that jointly optimizes for task-specific answer correctness, structured output adherence, and penalization of incorrect tool usage, thereby encouraging both precise reasoning and efficient tool integration. Extensive evaluations across diverse knowledge-intensive, mathematical, and general language modeling tasks demonstrate that AutoTIR achieves superior overall performance, significantly outperforming baselines and exhibits superior generalization in tool-use behavior. These results highlight the promise of reinforcement learning in building truly generalizable and scalable TIR capabilities in LLMs. The code and data are available at https://github.com/weiyifan1023/AutoTIR.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 29 of its 69 references that are in the map. First posted on arXiv 29 Jul 2025.

Builds on

29 of 69 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents18 Jun
  2. 2025Xolver: Multi-Agent Reasoning with Holistic Experience Learning Just Like an Olympiad Team17 Jun
  3. 2025Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning22 May
  4. 2025Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving12 May
  5. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMs15 Apr

Other references · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  6. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  7. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  8. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  9. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  10. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  11. 2024Large Language Models: A Surveycited by 525
  12. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
12 more
  1. 2025Process Reinforcement through Implicit Rewardscited by 504
  2. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
  3. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 282
  4. 2025ToRL: Scaling Tool-Integrated RLcited by 169
  5. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learningcited by 161
  6. 2025When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMscited by 53
  7. 2025MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learningcited by 48
  8. 2024MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoningcited by 34
  9. 2025Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agentcited by 30
  10. 2025Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Modelscited by 28
  11. 2023MenatQA: A New Dataset for Testing the Temporal Comprehension and Reasoning Abilities of Large Language Modelscited by 15
  12. 2025Structural Entropy Guided Agent for Detecting and Repairing Knowledge Deficiencies in LLMscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 21.

33 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  3. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 80
  4. 2025Demystifying Reinforcement Learning in Agentic Reasoningcited by 33
  5. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 19
  6. 2025AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agentcited by 17
  7. 2026Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuningcited by 17
  8. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 16
  9. 2025Learning How to Use Tools, Not Just When: Pattern-Aware Tool-Integrated Reasoningcited by 15
  10. 2025Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive Exploration for Agentic Reinforcement Learningcited by 11
  11. 2026Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Modelscited by 10
  12. 2026MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite Matchingcited by 10
21 more
  1. 2025THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoningcited by 9
  2. 2025Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AIcited by 8
  3. 2026SkillMaster: Toward Autonomous Skill Mastery in LLM Agentscited by 8
  4. 2025A2FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoningcited by 6
  5. 2026Learning from the Irrecoverable: Error-Localized Policy Optimization for Tool-Integrated LLM Reasoningcited by 6
  6. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  7. 2025PORTool: Importance-Aware Policy Optimization with Rewarded Tree for Multi-Tool-Integrated Reasoningcited by 4
  8. 2026ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibrationcited by 4
  9. 2026Learning When Not to Act: Mitigating Tool Abuse in Agentic Reinforcement Learningcited by 4
  10. 2026Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoningcited by 3
  11. 2026See Further, Think Deeper: Advancing VLM's Reasoning Ability with Low-level Visual Cues and Reflectioncited by 3
  12. 2026Discovery and Reinforcement of Tool-Integrated Reasoning Chains via Rollout Treescited by 2
  13. 2026Guided by Trajectories: Repairing and Rewarding Tool-Use Trajectories for Tool-Integrated Reasoningcited by 2
  14. 2026Tools as Continuous Flow for Evolving Agentic Reasoningcited by 2
  15. 2026Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learningcited by 2
  16. 2026TL-GRPO: Turn-Level RL for Reasoning-Guided Iterative Optimizationcited by 1
  17. 2026TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoningcited by 1
  18. 2025SIGMA: Search-Augmented On-Demand Knowledge Integration for Agentic Mathematical Reasoningcited by 0
  19. 2026AnomalyAgent: Agentic Industrial Anomaly Synthesis via Tool-Augmented Reinforcement Learningcited by 0
  20. 2026TIDE-Bench: Task-Aware and Diagnostic Evaluation of Tool-Integrated Reasoningcited by 0
  21. 2026DeepTool: Scaling Interleaved Deliberation in Tool-Integrated Reasoning via Process-Supervised Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning
  2. 2026TInR: Exploring Tool-Internalized Reasoning in Large Language Models
  3. 2026Are Tools Always Beneficial? Learning to Invoke Tools Adaptively for Dual-Mode Multimodal LLM Reasoning
  4. 2025Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning
  5. 2026AdaTIR: Adaptive Tool-Integrated Reasoning via Difficulty-Aware Policy Optimization
  6. 2025Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use
  7. 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning
  8. 2026Joint Optimization of Tool Creation and Use for Large Language Model Agents