# How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests

arXiv: https://arxiv.org/abs/2607.21832 · HTML: https://mlsanity.com/paper/2607.21832/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 45 of its 122 references that are in the map.
- First posted on arXiv 23 Jul 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 30 of 45)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [GitHub Copilot AI pair programmer: Asset or Liability?](https://mlsanity.com/paper/2206.15331/) (2022)
- [Gemini Embedding: Generalizable Embeddings from Gemini](https://mlsanity.com/paper/2503.07891/) (2025)
- [Automated Unit Test Improvement using Large Language Models at Meta](https://mlsanity.com/paper/2402.09171/) (2024)
- [Bugs in Large Language Models Generated Code: An Empirical Study](https://mlsanity.com/paper/2403.08937/) (2024)
- [On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub](https://mlsanity.com/paper/2509.14745/) (2025)
- [Using Large Language Models to Generate JUnit Tests: An Empirical Study](https://mlsanity.com/paper/2305.00418/) (2023)
- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [Agentic Software Engineering: Foundational Pillars and a Research Roadmap](https://mlsanity.com/paper/2509.06216/) (2025)
- [Evaluating Word Embedding Models: Methods and Experimental Results](https://mlsanity.com/paper/1901.09785/) (2019)
- [Agentic Much? Adoption of Coding Agents on GitHub](https://mlsanity.com/paper/2601.18341/) (2026)
- [HumanEvo: An Evolution-aware Benchmark for More Realistic Evaluation of Repository-level Code Generation](https://mlsanity.com/paper/2406.06918/) (2024)
- [Understanding the Factors that Impact the Popularity of GitHub Repositories](https://mlsanity.com/paper/1606.04984/) (2016)
- [What's in a GitHub Star? Understanding Repository Starring Practices in a Social Coding Platform](https://mlsanity.com/paper/1811.07643/) (2018)
- [Contextualized Topic Coherence Metrics](https://mlsanity.com/paper/2305.14587/) (2023)
- [An Empirical Study on Challenges for LLM Application Developers](https://mlsanity.com/paper/2408.05002/) (2024)
- [Agentic Refactoring: An Empirical Study of AI Coding Agents](https://mlsanity.com/paper/2511.04824/) (2025)
- [Bugs in Machine Learning-based Systems: A Faultload Benchmark](https://mlsanity.com/paper/2206.12311/) (2022)
- [SWE-Bench-CL: Continual Learning for Coding Agents](https://mlsanity.com/paper/2507.00014/) (2025)
- [Automating Dependency Updates in Practice: An Exploratory Study on GitHub Dependabot](https://mlsanity.com/paper/2206.07230/) (2022)
- [Is Your LLM Overcharging You? Tokenization, Transparency, and Incentives](https://mlsanity.com/paper/2505.21627/) (2025)
- [How Bad Can a Bug Get? An Empirical Analysis of Software Failures in the OpenStack Cloud Computing Platform](https://mlsanity.com/paper/1907.04055/) (2019)
- [AI IDEs or Autonomous Agents? Measuring the Impact of Coding Agents on Software Development](https://mlsanity.com/paper/2601.13597/) (2026)
- [Bug Characterization in Machine Learning-based Systems](https://mlsanity.com/paper/2307.14512/) (2023)
- [Does Code Quality Affect Pull Request Acceptance? An empirical study](https://mlsanity.com/paper/1908.09321/) (2019)
- [How AI Coding Agents Modify Code: A Large-Scale Study of GitHub Pull Requests](https://mlsanity.com/paper/2601.17581/) (2026)
- [More Code, Less Reuse: Investigating Code Quality and Reviewer Sentiment towards AI-generated Pull Requests](https://mlsanity.com/paper/2601.21276/) (2026)
- [Modern Code Reviews -- Survey of Literature and Practice](https://mlsanity.com/paper/2403.00088/) (2024)

## Abstract (arXiv)

Recent advances in large language models and their rapid adoption across software engineering tasks have made Artificial Intelligence (AI) coding agents an integral component of modern software development workflows. While developers increasingly benefit from these coding agents, their impact on software quality remains insufficiently understood. In particular, how agentic contributions evolve across the software development lifecycle has not been thoroughly investigated. This study aims to characterize agentic pull requests (PR) in comparison to human generated PRs and to examine how their properties change across different stages of the development lifecycle. Using the AIDev dataset, we first analyze how differences in merge rates between agentic and human generated PRs vary over time. We then identify the types of development tasks where AI coding agents are predominantly applied and investigate how these task distributions evolve across development quarters. Finally, we compare a set of key characteristics of agentic and human generated PRs, focusing on their implications for software quality and their temporal dynamics. Overall, our findings provide an empirical and longitudinal perspective on the role of AI coding agents in software development, offering a more nuanced understanding of their benefits and limitations in real-world practices.