# MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite Matching

arXiv: https://arxiv.org/abs/2601.10712 · HTML: https://mlsanity.com/paper/2601.10712/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 15 Jan 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Reinforcement learning for LLM agents](https://mlsanity.com/agents/reinforcement-learning-for-llm-agents/)

## Builds on (most cited first, 19 of 19)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Group Sequence Policy Optimization](https://mlsanity.com/paper/2507.18071/) (2025)
- [ReTool: Reinforcement Learning for Strategic Tool Use in LLMs](https://mlsanity.com/paper/2504.11536/) (2025)
- [DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments](https://mlsanity.com/paper/2504.03160/) (2025)
- [Agentic Reinforced Policy Optimization](https://mlsanity.com/paper/2507.19849/) (2025)
- [ToRL: Scaling Tool-Integrated RL](https://mlsanity.com/paper/2503.23383/) (2025)
- [SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning](https://mlsanity.com/paper/2509.02479/) (2025)
- [VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use](https://mlsanity.com/paper/2509.01055/) (2025)
- [SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attribution](https://mlsanity.com/paper/2505.20732/) (2025)
- [Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning](https://mlsanity.com/paper/2505.00024/) (2025)
- [Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agents](https://mlsanity.com/paper/2510.14967/) (2025)
- [Understanding Tool-Integrated Reasoning](https://mlsanity.com/paper/2508.19201/) (2025)
- [AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2507.21836/) (2025)
- [CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic](https://mlsanity.com/paper/2511.12159/) (2025)
- [Promoting Efficient Reasoning with Verifiable Stepwise Reward](https://mlsanity.com/paper/2508.10293/) (2025)
- [Exploiting Tree Structure for Credit Assignment in RL Training of LLMs](https://mlsanity.com/paper/2509.18314/) (2025)
- [Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments](https://mlsanity.com/paper/2508.08791/) (2025)
- [THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning](https://mlsanity.com/paper/2509.13761/) (2025)

## Built on by (most cited first, 10 of 10)

- [Recursive Self-Evolving Agents via Held-Out Selection](https://mlsanity.com/paper/2606.28374/) (2026)
- [Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple Actions](https://mlsanity.com/paper/2604.07277/) (2026)
- [PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agents](https://mlsanity.com/paper/2606.16215/) (2026)
- [DART-SD: Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents](https://mlsanity.com/paper/2608.18524/) (2026)
- [CRISP: Critical Step Perception for Training Efficient Deep Search Agents](https://mlsanity.com/paper/2608.01867/) (2026)
- [Cross-View Correspondence Is a Measurement Intervention: Two-Sided Validation for Agent Evaluation and Credit Assignment](https://mlsanity.com/paper/2608.17713/) (2026)
- [HiDiffTIR: Hierarchical Difficulty-Aware Policy Optimization for Multi-Turn Tool-Integrated Reasoning](https://mlsanity.com/paper/2608.21863/) (2026)
- [One Policy Is Enough: Single-Agent Reinforcement Learning Outperforms Tree Search for Chemistry Tool Learning](https://mlsanity.com/paper/2608.30952/) (2026)
- [Toolcompass: Guiding Tool Trialing, Not Suppressing It](https://mlsanity.com/paper/2609.25678/) (2026)
- [Learning after Execution with Action-Specific Credit](https://mlsanity.com/paper/2610.14573/) (2026)

## Abstract (arXiv)

Tool-Integrated Reasoning (TIR) empowers large language models (LLMs) to tackle complex tasks by interleaving reasoning steps with external tool interactions. However, existing reinforcement learning methods typically rely on outcome- or trajectory-level rewards, assigning uniform advantages to all steps within a trajectory. This coarse-grained credit assignment fails to distinguish effective tool calls from redundant or erroneous ones, particularly in long-horizon multi-turn scenarios. To address this, we propose MatchTIR, a framework that introduces fine-grained supervision via bipartite matching-based turn-level reward assignment and dual-level advantage estimation. Specifically, we formulate credit assignment as a bipartite matching problem between predicted and ground-truth traces, utilizing two assignment strategies to derive dense turn-level rewards. Furthermore, to balance local step precision with global task success, we introduce a dual-level advantage estimation scheme that integrates turn-level and trajectory-level signals, assigning distinct advantage values to individual interaction turns. Extensive experiments on three benchmarks demonstrate the superiority of MatchTIR. Notably, our 4B model surpasses the majority of 8B competitors, particularly in long-horizon and multi-turn tasks. Our codes are available at https://github.com/quchangle1/MatchTIR.