Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuning
On the map: LLM reasoning → Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
Agentic Reinforcement Learning (ARL) trains large language models to interleave reasoning with external tool execution to solve complex tasks. Most existing ARL methods train a single set of parameters to support both reasoning and tool-use behaviors, implicitly assuming that joint training leads to improved overall agent performance. Despite its widespread adoption, this assumption has rarely been examined empirically. In this paper, we systematically examine this assumption by introducing Capability Effect Attribution (CEA), which provides quantitative evidence of interference between reasoning and tool-use behaviors. Through an in-depth analysis, we show that these two capabilities often induce misaligned gradient directions, leading to training interference that undermines the effectiveness of joint optimization and challenges the prevailing ARL paradigm. To address this issue, we propose Disentangled Action--Reasoning Tuning (DART), a simple and efficient framework that explicitly decouples parameter updates for reasoning and tool use via separate low-rank adaptation modules. With this simple change alone, DART outperforms all joint-optimization baselines and approaches the 2-Agent upper bound across thirteen benchmarks on retrieval-augmented QA and NL2SQL, further supporting our finding of capability interference under shared optimization.
Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 23 of its 66 references that are in the map. First posted on arXiv 1 Feb 2026.
Builds on
23 of 66 references.
Closest ancestors · 5
Same direction, newest first
- 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use1 Sep
- 2025AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learning29 Jul
- 2025Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning22 May
- 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning28 Apr
- 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMs15 Apr
Other references · 18
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2024Qwen2 Technical Reportcited by 2,076
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
- 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
- 2025ToolRL: Reward is All Tool Learning Needscited by 342
- 2024MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Expertscited by 144
- 2025WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agentscited by 66
- 2025WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learningcited by 59
- 2024MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Modelscited by 54
- 2025DeepAnalyze: Agentic Large Language Models for Autonomous Data Sciencecited by 49
6 more
- 2023SiRA: Sparse Mixture of Low Rank Adaptationcited by 32
- 2025Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewardscited by 25
- 2025MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capabilitycited by 22
- 2025RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Usecited by 17
- 2026Automated Optimization Modeling via a Localizable Error-Driven Perspectivecited by 14
- 2026Towards a Theoretical Understanding to the Generalization of RLHFcited by 5
Built on by
Citing papers per year: 2026: 15.
15 papers.
Sort:
By area:
- 2026UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillationcited by 10
- 2026A Formula-Driven Survey and Research Agenda for On-Policy Distillationcited by 5
- 2026Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agentscited by 3
- 2026SocraticPO: Policy Optimization via Interactive Guidancecited by 3
- 2026Your "Pro" LLM Subscription May Actually Be "Free": Exposing Fingerprint Spoofing Risks in LLM Inference Servicescited by 3
- 2026Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signalscited by 2
- 2026HTAM: Hierarchical Transition-Attended Memory for Operator Optimizationcited by 1
- 2026VistaHop: Benchmarking Long-Horizon Visual DeepSearchcited by 1
- 2026Think Big, Search Small: Where Capacity Matters in Hierarchical Search Agents?cited by 1
- 2026ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capabilitycited by 1
- 2026M2A: Synergizing Mathematical and Agentic Reasoning in Large Language Modelscited by 0
- 2026Draw2Think: Harnessing Geometry Reasoning through Constraint Engine Interactioncited by 0
- 2026BRIDGE: Bilevel Retrieval-Credit-Aware Agentic Reinforcement Learning29 Sep 2026
- 2026Behavior Quotient Learning for Low-Rank Adaptation of LLM Agents11 Sep 2026
- 2026ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capability14 Jul 2026
- 2026Think Big, Search Small: Where Capacity Matters in Hierarchical Search Agents?8 Jul 2026
- 2026Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment6 Jul 2026
- 2026UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillation28 Jun 2026
- 2026A Formula-Driven Survey and Research Agenda for On-Policy Distillation22 Jun 2026
- 2026Your "Pro" LLM Subscription May Actually Be "Free": Exposing Fingerprint Spoofing Risks in LLM Inference Services15 Jun 2026
- 2026Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signals9 Jun 2026
- 2026SocraticPO: Policy Optimization via Interactive Guidance3 Jun 2026
- 2026VistaHop: Benchmarking Long-Horizon Visual DeepSearch2 Jun 2026
- 2026HTAM: Hierarchical Transition-Attended Memory for Operator Optimization28 May 2026
- 2026UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillationcited by 10
- 2026Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agentscited by 3
- 2026Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signalscited by 2
- 2026ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capabilitycited by 1
- 2026Behavior Quotient Learning for Low-Rank Adaptation of LLM Agentscited by 0
- 2026BRIDGE: Bilevel Retrieval-Credit-Aware Agentic Reinforcement Learningcited by 0
- 2026BRIDGE: Bilevel Retrieval-Credit-Aware Agentic Reinforcement Learning29 Sep 2026
- 2026Behavior Quotient Learning for Low-Rank Adaptation of LLM Agents11 Sep 2026
- 2026ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capability14 Jul 2026
- 2026UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillation28 Jun 2026
- 2026Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signals9 Jun 2026
- 2026Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agents23 May 2026
- 2026SocraticPO: Policy Optimization via Interactive Guidancecited by 3
- 2026Think Big, Search Small: Where Capacity Matters in Hierarchical Search Agents?cited by 1
- 2026M2A: Synergizing Mathematical and Agentic Reasoning in Large Language Modelscited by 0
- 2026Draw2Think: Harnessing Geometry Reasoning through Constraint Engine Interactioncited by 0
- 2026Think Big, Search Small: Where Capacity Matters in Hierarchical Search Agents?8 Jul 2026
- 2026SocraticPO: Policy Optimization via Interactive Guidance3 Jun 2026
- 2026Draw2Think: Harnessing Geometry Reasoning through Constraint Engine Interaction20 May 2026
- 2026M2A: Synergizing Mathematical and Agentic Reasoning in Large Language Models11 May 2026
- 2026VistaHop: Benchmarking Long-Horizon Visual DeepSearchcited by 1
- 2026VistaHop: Benchmarking Long-Horizon Visual DeepSearch2 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act
- 2026ASTER: Agentic Scaling with Tool-integrated Extended Reasoning
- 2026Does RL Expand the Capability Boundary of LLM Agents? A PASS@(k,T) Analysis
- 2025Demystifying Reinforcement Learning in Agentic Reasoning
- 2025AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning
- 2026Tool-Aware Optimization with Entropy Guidance for Efficient Agentic Reinforcement Learning
- 2026On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
- 2026Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancement