# Pull Requests as a Training Signal for Repo-Level Code Editing

arXiv: https://arxiv.org/abs/2602.07457 · HTML: https://mlsanity.com/paper/2602.07457/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 7 Feb 2026.

## Builds on (most cited first, 25 of 25)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [SWE-smith: Scaling Data for Software Engineering Agents](https://mlsanity.com/paper/2504.21798/) (2025)
- [AutoCodeRover: Autonomous Program Improvement](https://mlsanity.com/paper/2404.05427/) (2024)
- [OctoPack: Instruction Tuning Code Large Language Models](https://mlsanity.com/paper/2308.07124/) (2023)
- [Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving](https://mlsanity.com/paper/2504.02605/) (2025)
- [SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents](https://mlsanity.com/paper/2505.20411/) (2025)
- [SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement](https://mlsanity.com/paper/2410.20285/) (2024)
- [Automating Code Review Activities by Large-Scale Pre-training](https://mlsanity.com/paper/2203.09095/) (2022)
- [SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution](https://mlsanity.com/paper/2501.05040/) (2025)
- [SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents](https://mlsanity.com/paper/2504.08703/) (2025)
- [SWE-bench Goes Live!](https://mlsanity.com/paper/2505.23419/) (2025)
- [Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models](https://mlsanity.com/paper/2403.04811/) (2024)
- [CoditT5: Pretraining for Source Code and Natural Language Editing](https://mlsanity.com/paper/2208.05446/) (2022)
- [Issue Localization via LLM-Driven Iterative Code Graph Searching](https://mlsanity.com/paper/2503.22424/) (2025)
- [SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugs](https://mlsanity.com/paper/2504.14757/) (2025)
- [What Makes a Good Commit Message?](https://mlsanity.com/paper/2202.02974/) (2022)
- [Coeditor: Leveraging Contextual Changes for Multi-round Code Auto-editing](https://mlsanity.com/paper/2305.18584/) (2023)
- [CommitBench: A Benchmark for Commit Message Generation](https://mlsanity.com/paper/2403.05188/) (2024)
- [Towards A Generalist Code Embedding Model Based On Massive Data Synthesis](https://mlsanity.com/paper/2505.12697/) (2025)

## Built on by (most cited first, 1 of 1)

- [Learning to Commit: Generating Organic Pull Requests via Online Repository Memory](https://mlsanity.com/paper/2603.26664/) (2026)

## Abstract (arXiv)

Repository-level code editing requires models to understand complex dependencies and execute precise multi-file modifications across a large codebase. While recent gains on SWE-bench rely heavily on complex agent scaffolding, it remains unclear how much of this capability can be internalised via high-quality training signals. To address this, we propose Clean Pull Request (Clean-PR), a mid-training paradigm that leverages real-world GitHub pull requests as a training signal for repository-level editing. We introduce a scalable pipeline that converts noisy pull request diffs into Search/Replace edit blocks through reconstruction and validation, resulting in the largest publicly available corpus of 2 million pull requests spanning 12 programming languages. Using this training signal, we perform a mid-training stage followed by an agentless-aligned supervised fine-tuning process with error-driven data augmentation. On SWE-bench, our model significantly outperforms the instruction-tuned baseline, achieving absolute improvements of 13.6% on SWE-bench Lite and 12.3% on SWE-bench Verified. These results demonstrate that repository-level code understanding and editing capabilities can be effectively internalised into model weights under a simplified, agentless protocol, without relying on heavy inference-time scaffolding.