How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests

arXiv 2607.21832 ↗First posted 23 Jul 2026cs.SE cs.LGcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Recent advances in large language models and their rapid adoption across software engineering tasks have made Artificial Intelligence (AI) coding agents an integral component of modern software development workflows. While developers increasingly benefit from these coding agents, their impact on software quality remains insufficiently understood. In particular, how agentic contributions evolve across the software development lifecycle has not been thoroughly investigated. This study aims to characterize agentic pull requests (PR) in comparison to human generated PRs and to examine how their properties change across different stages of the development lifecycle. Using the AIDev dataset, we first analyze how differences in merge rates between agentic and human generated PRs vary over time. We then identify the types of development tasks where AI coding agents are predominantly applied and investigate how these task distributions evolve across development quarters. Finally, we compare a set of key characteristics of agentic and human generated PRs, focusing on their implications for software quality and their temporal dynamics. Overall, our findings provide an empirical and longitudinal perspective on the role of AI coding agents in software development, offering a more nuanced understanding of their benefits and limitations in real-world practices.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 45 of its 122 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

45 of 122 references.

Closest ancestors · 11

Same direction, newest first

Other references · 34

2021Evaluating Large Language Models Trained on Codecited by 8,320
2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 152
2025Gemini Embedding: Generalizable Embeddings from Geminicited by 94
2024Automated Unit Test Improvement using Large Language Models at Metacited by 75
2024Bugs in Large Language Models Generated Code: An Empirical Studycited by 60
2023Using Large Language Models to Generate JUnit Tests: An Empirical Studycited by 53
2019Evaluating Word Embedding Models: Methods and Experimental Resultscited by 37
2024HumanEvo: An Evolution-aware Benchmark for More Realistic Evaluation of Repository-level Code Generationcited by 28
2016Understanding the Factors that Impact the Popularity of GitHub Repositoriescited by 27
2018What's in a GitHub Star? Understanding Repository Starring Practices in a Social Coding Platformcited by 25

22 more

2023Contextualized Topic Coherence Metricscited by 19
2024An Empirical Study on Challenges for LLM Application Developerscited by 18
2022Bugs in Machine Learning-based Systems: A Faultload Benchmarkcited by 15
2025SWE-Bench-CL: Continual Learning for Coding Agentscited by 14
2022Automating Dependency Updates in Practice: An Exploratory Study on GitHub Dependabotcited by 13
2025Is Your LLM Overcharging You? Tokenization, Transparency, and Incentivescited by 13
2019How Bad Can a Bug Get? An Empirical Analysis of Software Failures in the OpenStack Cloud Computing Platformcited by 11
2023Bug Characterization in Machine Learning-based Systemscited by 10
2019Does Code Quality Affect Pull Request Acceptance? An empirical studycited by 9
2024Modern Code Reviews -- Survey of Literature and Practicecited by 6
2026Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomycited by 6
2020Are Multi-language Design Smells Fault-prone? An Empirical Studycited by 5
2025Intuition to Evidence: Measuring AI's True Impact on Developer Productivitycited by 5
2026On Autopilot? An Empirical Study of Human-AI Teaming and Review Practices in Open Sourcecited by 5
2022Studying the Characteristics of AIOps Projects on GitHubcited by 4
2023How do Software Engineering Researchers Use GitHub? An Empirical Study of Artifacts & Impactcited by 3
2024Empirical Analysis on CI/CD Pipeline Evolution in Machine Learning Projectscited by 3
2026The Productivity-Reliability Paradox: Specification-Driven Governance for AI-Augmented Software Developmentcited by 3
2022Towards Understanding Third-party Library Dependency in C/C++ Ecosystemcited by 2
2023Validation of Modern JSON Schema: Formalization and Complexitycited by 2
2024Chronicles of CI/CD: A Deep Dive into its Usage Over Timecited by 1
2024Empirical Analysis of Pull Requests for Google Summer of Codecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way