# SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning

arXiv: https://arxiv.org/abs/2502.20127 · HTML: https://mlsanity.com/paper/2502.20127/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 28 of 28)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Executable Code Actions Elicit Better LLM Agents](https://mlsanity.com/paper/2402.01030/) (2024)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [ReFT: Reasoning with Reinforced Fine-Tuning](https://mlsanity.com/paper/2401.08967/) (2024)
- [Rule Based Rewards for Language Model Safety](https://mlsanity.com/paper/2411.01111/) (2024)
- [SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution](https://mlsanity.com/paper/2501.05040/) (2025)
- [Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement](https://mlsanity.com/paper/2411.00622/) (2024)
- [Commit0: Library Generation from Scratch](https://mlsanity.com/paper/2412.01769/) (2024)
- [RepoQA: Evaluating Long Context Code Understanding](https://mlsanity.com/paper/2406.06025/) (2024)
- [Compositional API Recommendation for Library-Oriented Code Generation](https://mlsanity.com/paper/2402.19431/) (2024)
- [Repository Structure-Aware Training Makes SLMs Better Issue Resolver](https://mlsanity.com/paper/2412.19031/) (2024)

## Built on by (most cited first, 28 of 28)

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains](https://mlsanity.com/paper/2503.23829/) (2025)
- [WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch](https://mlsanity.com/paper/2505.03733/) (2025)
- [SWE-Exp: Experience-Driven Software Issue Resolution](https://mlsanity.com/paper/2507.23361/) (2025)
- [One Token to Fool LLM-as-a-Judge](https://mlsanity.com/paper/2507.08794/) (2025)
- [Agentic Software Engineering: Foundational Pillars and a Research Roadmap](https://mlsanity.com/paper/2509.06216/) (2025)
- [Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute](https://mlsanity.com/paper/2503.23803/) (2025)
- [Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches](https://mlsanity.com/paper/2510.04905/) (2025)
- [Issue Localization via LLM-Driven Iterative Code Graph Searching](https://mlsanity.com/paper/2503.22424/) (2025)
- [Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks](https://mlsanity.com/paper/2512.03262/) (2025)
- [Tool-integrated Reinforcement Learning for Repo Deep Search](https://mlsanity.com/paper/2508.03012/) (2025)
- [WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning](https://mlsanity.com/paper/2509.22644/) (2025)
- [Co-PatcheR: Collaborative Software Patching with Component(s)-specific Small Reasoning Models](https://mlsanity.com/paper/2505.18955/) (2025)
- [Scaling Coding Agents via Atomic Skills](https://mlsanity.com/paper/2604.05013/) (2026)
- [SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks](https://mlsanity.com/paper/2506.10954/) (2025)
- [Agentic Software Issue Resolution with Large Language Models: A Survey](https://mlsanity.com/paper/2512.22256/) (2025)
- [How Well Do Large Reasoning Models Translate? A Comprehensive Evaluation for Multi-Domain Machine Translation](https://mlsanity.com/paper/2505.19987/) (2025)
- [Scalable Supervision for Software Agents via Patch Reasoning](https://mlsanity.com/paper/2510.22775/) (2025)
- [A Benchmark for Localizing Code and Non-Code Issues in Software Projects](https://mlsanity.com/paper/2509.25242/) (2025)
- [From Historical Patches to Repair Plans: Outcome-Conditioned Reasoning for Repository-Level Program Repair](https://mlsanity.com/paper/2601.23257/) (2026)
- [FullStack-Agent: Enhancing Agentic Full-Stack Web Coding via Development-Oriented Testing and Repository Back-Translation](https://mlsanity.com/paper/2602.03798/) (2026)
- [BitsAI-Fix: LLM-Driven Approach for Automated Lint Error Resolution in Practice](https://mlsanity.com/paper/2508.03487/) (2025)
- [ORACLE-SWE: Quantifying the Contribution of Oracle Information Signals on SWE Agents](https://mlsanity.com/paper/2604.07789/) (2026)
- [Neurosymbolic Repo-level Code Localization](https://mlsanity.com/paper/2604.16021/) (2026)
- [SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning](https://mlsanity.com/paper/2601.21649/) (2026)
- [Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents](https://mlsanity.com/paper/2609.33875/) (2026)
- [SecureVibe: Making Vibe Coding More Secure](https://mlsanity.com/paper/2609.38606/) (2026)
- [Teaching Agents to Code Reliably](https://mlsanity.com/paper/2610.03984/) (2026)

## Abstract (arXiv)

Mainstream issue-resolving frameworks predominantly rely on commercial models, leading to high costs and privacy concerns. Existing training approaches for issue resolving struggle with poor generalization and fail to fully leverage open-source development resources. We propose Subtask-oriented Reinforced Fine-Tuning (SoRFT), a novel training approach to enhance the issue resolving capability of LLMs. We decomposes issue resolving into structured subtasks: file localization, function localization, line localization, and code edit generation. SoRFT consists of two training stages: (1) rejection-sampled supervised fine-tuning, Chain of Thought (CoT) data is filtered using ground-truth before fine-tuning the LLM, and (2) rule-based reinforcement learning, which leverages PPO with ground-truth based rewards. We evaluate the SoRFT-trained model on SWE-Bench Verified and SWE-Bench Lite, achieving state-of-the-art (SOTA) performance among open-source models (e.g., resolve 21.4% issues on SWE-Bench Verified with SoRFT-Qwen-7B). The experimental results demonstrate that SoRFT significantly enhances issue-resolving performance, improves model generalization, and provides a cost-efficient alternative to commercial models.