mlsanityData to 7 Oct 2026

Why Are AI Agent Involved Pull Requests (Fix-Related) Remain Unmerged? An Empirical Study

arXiv 2602.00164 ↗First posted 29 Jan 2026cs.SE cs.AIcited by 3

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Autonomous coding agents (e.g., OpenAI Codex, Devin, GitHub Copilot) are increasingly used to generate fix-related pull requests (PRs) in real world software repositories. However, their practical effectiveness depends on whether these contributions are accepted and merged by project maintainers. In this paper, we present an empirical study of AI agent involved fix related PRs, examining both their integration outcomes, latency, and the factors that hinder successful merging. We first analyze 8,106 fix related PRs authored by five widely used AI coding agents from the AIDEV POP dataset to quantify the proportions of PRs that are merged, closed without merging, or remain open. We then conduct a manual qualitative analysis of a statistically significant sample of 326 closed but unmerged PRs, spending approximately 100 person hours to construct a structured catalog of 12 failure reasons. Our results indicate that test case failures and prior resolution of the same issues by other PRs are the most common causes of non integration, whereas build or deployment failures are comparatively rare. Overall, our findings expose key limitations of current AI coding agents in real world settings and highlight directions for their further improvement and for more effective human AI collaboration in software maintenance.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 11 of its 22 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

11 of 22 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub18 Sep
  2. 2024Agents in Software Engineering: Survey, Landscape, and Vision13 Sep
  3. 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repair25 Mar

Other references · 8

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  2. 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 264
  3. 2023An Analysis of the Automatic Bug Fixing Performance of ChatGPTcited by 108
  4. 2024Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmapcited by 30
  5. 2025AI in Software Engineering: Perceived Roles and Their Impact on Adoptioncited by 5
  6. 2014A Critical Review of "Automatic Patch Generation Learned from Human-Written Patches": Essay on the Problem Statement and the Evaluation of Automatic Software Repaircited by 4
  7. 2022Automatic Prediction of Rejected Edits in Stack Overflowcited by 4
  8. 2025What Drives Issue Resolution Speed? An Empirical Study of Scientific Workflow Systems on GitHubcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub
  2. 2026Understanding the Rejection of Fixes Generated by Agentic Pull Requests -- Insights from the AIDev Dataset
  3. 2026Beyond Bug Fixes: An Empirical Investigation of Post-Merge Code Quality Issues in Agent-Generated Pull Requests
  4. 2026Why Are Agentic Pull Requests Merged or Rejected? An Empirical Study
  5. 2026Not All Agents Are Equal: Code Quality and Post-Merge Maintenance Across Five Autonomous Coding Agents in the Wild
  6. 2026How AI Coding Agents Modify Code: A Large-Scale Study of GitHub Pull Requests
  7. 2026AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub
  8. 2026Safer Builders, Risky Maintainers: A Comparative Study of Breaking Changes in Human vs Agentic PRs