# Merge-Bench: Resolve Merge Conflicts with Large Language Models

arXiv: https://arxiv.org/abs/2605.25890 · HTML: https://mlsanity.com/paper/2605.25890/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 25 May 2026.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Program Merge Conflict Resolution via Neural Transformers](https://mlsanity.com/paper/2109.00084/) (2021)
- [GitGoodBench: A Novel Benchmark For Evaluating Agentic Performance On Git](https://mlsanity.com/paper/2505.22583/) (2025)
- [DeepMerge: Learning to Merge Programs](https://mlsanity.com/paper/2105.07569/) (2021)
- [Spork: Structured Merge for Java with Formatting Preservation](https://mlsanity.com/paper/2202.05329/) (2022)
- [CONGRA: Benchmarking Automatic Conflict Resolution](https://mlsanity.com/paper/2409.14121/) (2024)
- [Evaluation of Version Control Merge Tools](https://mlsanity.com/paper/2410.09934/) (2024)

## Abstract (arXiv)

This paper applies machine learning to the difficult and important task of version control merging. (1) We constructed a dataset, Merge-Bench, of 7938 real-world merge conflict hunks from 1439 GitHub repositories. The ground truth is the merge resolution that developers committed to the repository. Our dataset construction methodology is scalable to arbitrary amounts of data since no manual labeling is required. (2) We trained a model, LLMergeJ, to resolve merge conflicts in Java programs. Our approach uses Group Relative Policy Optimization (GRPO), an online reinforcement learning method, to train a Large Language Model (LLM). (3) We performed two evaluations of the performance of LLMs on resolving merge conflicts. On Java programs, LLMergeJ with 14B parameters outperforms 3 commercial LLMs, trailing only Gemini 2.5 Pro. Across 11 programming languages, commercial LLM performance is largely stable from language to language. The best models correctly resolve less than 60% of merge conflicts.