# Tool-integrated Reinforcement Learning for Repo Deep Search

arXiv: https://arxiv.org/abs/2508.03012 · HTML: https://mlsanity.com/paper/2508.03012/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 26 of 26)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training](https://mlsanity.com/paper/2501.17161/) (2025)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects](https://mlsanity.com/paper/2004.02806/) (2020)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [SWE-smith: Scaling Data for Software Engineering Agents](https://mlsanity.com/paper/2504.21798/) (2025)
- [Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning](https://mlsanity.com/paper/2502.14768/) (2025)
- [Monte Carlo Tree Search: A Review of Recent Modifications and Applications](https://mlsanity.com/paper/2103.04931/) (2021)
- [A Theoretical Analysis of NDCG Type Ranking Measures](https://mlsanity.com/paper/1304.6480/) (2013)
- [CodeR: Issue Resolving with Multi-Agent and Task Graphs](https://mlsanity.com/paper/2406.01304/) (2024)
- [LocAgent: Graph-Guided LLM Agents for Code Localization](https://mlsanity.com/paper/2503.09089/) (2025)
- [LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluation](https://mlsanity.com/paper/2404.10100/) (2024)
- [OrcaLoca: An LLM Agent Framework for Software Issue Localization](https://mlsanity.com/paper/2502.00350/) (2025)
- [HITS: High-coverage LLM-based Unit Test Generation via Method Slicing](https://mlsanity.com/paper/2408.11324/) (2024)
- [SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning](https://mlsanity.com/paper/2502.20127/) (2025)
- [Issue Localization via LLM-Driven Iterative Code Graph Searching](https://mlsanity.com/paper/2503.22424/) (2025)
- [Fault Localization with Code Coverage Representation Learning](https://mlsanity.com/paper/2103.00270/) (2021)
- [SEAlign: Alignment Training for Software Engineering Agent](https://mlsanity.com/paper/2503.18455/) (2025)

## Built on by (most cited first, 16 of 16)

- [Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches](https://mlsanity.com/paper/2510.04905/) (2025)
- [WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning](https://mlsanity.com/paper/2509.22644/) (2025)
- [Atlas: Orchestrating Heterogeneous Models and Tools for Multi-Domain Complex Reasoning](https://mlsanity.com/paper/2601.03872/) (2026)
- [VulnLLM-R: Specialized Reasoning LLM with Agent Scaffold for Vulnerability Detection](https://mlsanity.com/paper/2512.07533/) (2025)
- [Advances and Frontiers of LLM-based Issue Resolution in Software Engineering: A Comprehensive Survey](https://mlsanity.com/paper/2601.11655/) (2026)
- [CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents](https://mlsanity.com/paper/2603.17829/) (2026)
- [FastContext: Training Efficient Repository Explorer for Coding Agents](https://mlsanity.com/paper/2606.14066/) (2026)
- [CoSineVerifier: Tool-Augmented Answer Verification for Computation-Oriented Scientific Questions](https://mlsanity.com/paper/2512.01224/) (2025)
- [SHERLOC: Structured Diagnostic Localization for Code Repair Agents](https://mlsanity.com/paper/2606.24820/) (2026)
- [MM-IssueLoc: A Controlled Benchmark for Evaluating Visual Evidence in Multimodal Repository-Level Issue Localization](https://mlsanity.com/paper/2607.15205/) (2026)
- [PIPer: On-Device Environment Setup via Online Reinforcement Learning](https://mlsanity.com/paper/2509.25455/) (2025)
- [One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents](https://mlsanity.com/paper/2512.20957/) (2025)
- [A^2Agent: Action-Aware Reinforcement Learning for Repository-Level Code Localization Agents](https://mlsanity.com/paper/2608.29831/) (2026)
- [Semantic Navigation for Issue Localization in Code Repository](https://mlsanity.com/paper/2609.31176/) (2026)
- [PatchHolmes: Agentic Patch Retrieval via Listwise Selection](https://mlsanity.com/paper/2609.38807/) (2026)
- [Small Agents with Semantic Search: Efficient Multilingual Code Localization](https://mlsanity.com/paper/2610.05099/) (2026)

## Abstract (arXiv)

Issue localization, the process of identifying code locations that need modification to resolve software issues, is a critical yet challenging task in software development. The semantic gap between natural language issue descriptions and faulty code requires complex multi-hop reasoning through code dependencies. Existing LLM-based agents attempt to address this by integrating repository retrieval tools. However, this transforms issue localization into a demanding task we call Repo Deep Search, which requires the LLM to effectively utilize various repository retrieval tools throughout a multi-step reasoning and navigation process. To tackle this challenge, we present ToolTrain, a two-stage tool-integrated training framework combining rejection-sampled supervised fine-tuning and tool-integrated reinforcement learning to enhance LLMs' ability to use retrieval tools for issue localization. Experimental results show that ToolTrain-trained models achieve state-of-the-art performance, with our 32B model even surpassing Claude-3.7 on function-level localization. The results also show that improved localization performance translates to better end-to-end issue resolution performance. This further demonstrates that training for issue localization is a viable and effective strategy for improving automated software development.