# On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub

arXiv: https://arxiv.org/abs/2509.14745 · HTML: https://mlsanity.com/paper/2509.14745/

## Key facts

- Cited by 53 papers in the map (data through 7 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 30 of 33)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [Self-collaboration Code Generation via ChatGPT](https://mlsanity.com/paper/2304.07590/) (2023)
- [RepairAgent: An Autonomous, LLM-Based Agent for Program Repair](https://mlsanity.com/paper/2403.17134/) (2024)
- [LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead](https://mlsanity.com/paper/2404.04834/) (2024)
- [GitHub Copilot AI pair programmer: Asset or Liability?](https://mlsanity.com/paper/2206.15331/) (2022)
- [Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproduction](https://mlsanity.com/paper/2209.11515/) (2022)
- [Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Servers](https://mlsanity.com/paper/2506.13538/) (2025)
- [Towards Automating Code Review Activities](https://mlsanity.com/paper/2101.02518/) (2021)
- [Bias Testing and Mitigation in LLM-based Code Generation](https://mlsanity.com/paper/2309.14345/) (2023)
- [No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT](https://mlsanity.com/paper/2308.04838/) (2023)
- [Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Study](https://mlsanity.com/paper/2309.08221/) (2023)
- [On the Effectiveness of Large Language Models in Domain-Specific Code Generation](https://mlsanity.com/paper/2312.01639/) (2023)
- [Automatic Programming: Large Language Models and Beyond](https://mlsanity.com/paper/2405.02213/) (2024)
- [Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap](https://mlsanity.com/paper/2410.06107/) (2024)
- [How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study](https://mlsanity.com/paper/2101.06098/) (2021)
- [An Empirical Comparison of Pre-Trained Models of Source Code](https://mlsanity.com/paper/2302.04026/) (2023)
- [On Evaluating the Efficiency of Source Code Generated by LLMs](https://mlsanity.com/paper/2404.06041/) (2024)
- [Exploring the Capabilities of LLMs for Code Change Related Tasks](https://mlsanity.com/paper/2407.02824/) (2024)
- [Unveiling ChatGPT's Usage in Open Source Projects: A Mining-based Study](https://mlsanity.com/paper/2402.16480/) (2024)
- [The Role of Generative AI in Software Development Productivity: A Pilot Case Study](https://mlsanity.com/paper/2406.00560/) (2024)
- [Empowering Agile-Based Generative Software Development through Human-AI Teamwork](https://mlsanity.com/paper/2407.15568/) (2024)
- [The Product Beyond the Model -- An Empirical Study of Repositories of Open-Source ML Products](https://mlsanity.com/paper/2308.04328/) (2023)
- [Accountability in Code Review: The Role of Intrinsic Drivers and the Impact of LLMs](https://mlsanity.com/paper/2502.15963/) (2025)
- [Fight Fire with Fire: How Much Can We Trust ChatGPT on Source Code-Related Tasks?](https://mlsanity.com/paper/2405.12641/) (2024)
- [Motivations, Challenges, Best Practices, and Benefits for Bots and Conversational Agents in Software Engineering: A Multivocal Literature Review](https://mlsanity.com/paper/2409.11864/) (2024)
- [The Good, the Bad, and the Missing: Neural Code Generation for Machine Learning Tasks](https://mlsanity.com/paper/2305.09082/) (2023)
- [AutoPRTitle: A Tool for Automatic Pull Request Title Generation](https://mlsanity.com/paper/2206.11619/) (2022)
- [Are We Testing or Being Tested? Exploring the Practical Applications of Large Language Models in Software Testing](https://mlsanity.com/paper/2312.04860/) (2023)
- [Generative AI for Test Driven Development: Preliminary Results](https://mlsanity.com/paper/2405.10849/) (2024)

## Built on by (most cited first, 30 of 53)

- [Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projects](https://mlsanity.com/paper/2511.04427/) (2025)
- [Agentic Refactoring: An Empirical Study of AI Coding Agents](https://mlsanity.com/paper/2511.04824/) (2025)
- [Decoding the Configuration of AI Coding Agents: Insights from Claude Code Projects](https://mlsanity.com/paper/2511.09268/) (2025)
- [AI IDEs or Autonomous Agents? Measuring the Impact of Coding Agents on Software Development](https://mlsanity.com/paper/2601.13597/) (2026)
- [Semantic Soft Bootstrapping: Long Context Reasoning in LLMs without Reinforcement Learning](https://mlsanity.com/paper/2512.05105/) (2025)
- [How AI Coding Agents Modify Code: A Large-Scale Study of GitHub Pull Requests](https://mlsanity.com/paper/2601.17581/) (2026)
- [Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHub](https://mlsanity.com/paper/2601.00477/) (2026)
- [Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents](https://mlsanity.com/paper/2606.05391/) (2026)
- [What to Cut? Predicting Unnecessary Methods in Agentic Code Generation](https://mlsanity.com/paper/2602.17091/) (2026)
- [How Do Agents Perform Code Optimization? An Empirical Study](https://mlsanity.com/paper/2512.21757/) (2025)
- [Understanding Dominant Themes in Reviewing Agentic AI-authored Code](https://mlsanity.com/paper/2601.19287/) (2026)
- [Analyzing Message-Code Inconsistency in AI Coding Agent-Authored Pull Requests](https://mlsanity.com/paper/2601.04886/) (2026)
- [Are We All Using Agents the Same Way? An Empirical Study of Core and Peripheral Developers Use of Coding Agents](https://mlsanity.com/paper/2601.20106/) (2026)
- [Why Are AI Agent Involved Pull Requests (Fix-Related) Remain Unmerged? An Empirical Study](https://mlsanity.com/paper/2602.00164/) (2026)
- [Why Agentic-PRs Get Rejected: A Comparative Study of Coding Agents](https://mlsanity.com/paper/2602.04226/) (2026)
- [Human-AI Synergy in Agentic Code Review](https://mlsanity.com/paper/2603.15911/) (2026)
- [Investigating Autonomous Agent Contributions in the Wild: Activity Patterns and Code Change over Time](https://mlsanity.com/paper/2604.00917/) (2026)
- [A Dataset of Agentic AI Coding Tool Configurations](https://mlsanity.com/paper/2605.08435/) (2026)
- [Agentic AI in Industry: Adoption Level and Deployment Barriers](https://mlsanity.com/paper/2605.14675/) (2026)
- [Why Are Agentic Pull Requests Merged or Rejected? An Empirical Study](https://mlsanity.com/paper/2605.22534/) (2026)
- [Code for Machines, Not Just Humans: Quantifying AI-Friendliness with Code Health Metrics](https://mlsanity.com/paper/2601.02200/) (2026)
- [Code Change Characteristics and Description Alignment: A Comparative Study of Agentic versus Human Pull Requests](https://mlsanity.com/paper/2601.17627/) (2026)
- [Do AI Coding Agents Log Like Humans? An Empirical Study](https://mlsanity.com/paper/2604.09409/) (2026)
- [To What Extent Does Agent-generated Code Require Maintenance? An Empirical Study](https://mlsanity.com/paper/2605.06464/) (2026)
- [Agentic Agile-V: From Vibe Coding to Verified Engineering in Software and Hardware Development](https://mlsanity.com/paper/2605.20456/) (2026)
- [The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behavior](https://mlsanity.com/paper/2608.11386/) (2026)
- [User Misconceptions of LLM-Based Conversational Programming Assistants](https://mlsanity.com/paper/2510.25662/) (2025)
- [GLOW: Graph-Language Co-Encoding for Agentic Workflow Performance Prediction](https://mlsanity.com/paper/2512.15751/) (2025)
- [How Do Agentic AI Systems Address Performance Optimizations? A BERTopic-Based Analysis of Pull Requests](https://mlsanity.com/paper/2512.24630/) (2025)
- [On the Adoption of AI Coding Agents in Open-source Android and iOS Development](https://mlsanity.com/paper/2602.12144/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly being integrated into software development processes. The ability to generate code and submit pull requests with minimal human intervention, through the use of autonomous AI agents, is poised to become a standard practice. However, little is known about the practical usefulness of these pull requests and the extent to which their contributions are accepted in real-world projects. In this paper, we empirically study 567 GitHub pull requests (PRs) generated using Claude Code, an agentic coding tool, across 157 diverse open-source projects. Our analysis reveals that developers tend to rely on agents for tasks such as refactoring, documentation, and testing. The results indicate that 83.8% of these agent-assisted PRs are eventually accepted and merged by project maintainers, with 54.9% of the merged PRs are integrated without further modification. The remaining 45.1% require additional changes benefit from human revisions, especially for bug fixes, documentation, and adherence to project-specific standards. These findings suggest that while agent-assisted PRs are largely acceptable, they still benefit from human oversight and refinement.